Optional and Result, in Depth

Optional and Result in Volume 1 ended on three compile errors: unchecked access (E08030), a discarded result (E11052), and a check made stale by reassignment (E08040). This chapter is the machinery behind those three. If you know Java's Optional you will have assumed EK9's is the same thing with nicer syntax. It is not. Java's get() is legal anywhere and throws when you are wrong; EK9's get() is legal only inside a block whose check proved the value present, and the compiler, not a runtime check, is what stops you. This chapter is about that flow analysis: what it can see, what it refuses to see, and why the refusal is the feature.

Two ordinary objects

Optional of T holds zero or one value. Result of (O, E) holds an ok value, an error value, both, or neither; the last two are a surprise, and the chapter returns to them. Neither type is special to the type checker. They are library types, tri-state like every other EK9 object (set, unset or absent), with ? meaning "set". There is no unwrap, no orElseThrow, no force-access of any kind. The calls that matter are get, getOrDefault and whenPresent on Optional, and ok, error, isOk, isError, okOrDefault, errorOrDefault, getOrDefault, whenOk and whenError on Result.¹ Nothing in those types stops you calling get() on an empty Optional. What stops you is a pass over your control flow.

Here is a start-up routine that reads four settings. One lookup may find nothing; one parse may fail and says why.

#!ek9
defines module book.settings

  defines constant
    LOWEST_PORT <- 1024
    HIGHEST_PORT <- 65535

  defines function

    setting() as pure
      ->
        settings as Dict of (String, String)
        key as String
      <- rtn <- Optional() of String

      if settings contains key
        rtn: Optional(settings.getOrDefault(key, String()))

    parsePort() as pure
      -> text as String
      <- rtn as Result of (Integer, String)?

      port <- Integer(text)
      if not port?
        rtn: Result(Integer(), `'${text}' is not a whole number`)
      else if port < LOWEST_PORT or port > HIGHEST_PORT
        rtn: Result(Integer(), `${port} is outside ${LOWEST_PORT} ... ${HIGHEST_PORT}`)
      else
        rtn: Result(port, String())

    showPort()
      -> port as Integer
      Stdout().println(`listening on ${port}`)

    showProblem()
      -> problem as String
      Stdout().println(`cannot listen: ${problem}`)

  defines program

    StartServer()
      stdout <- Stdout()
      settings <- {"host": "example.org", "port": "8o80", "retries": "3"}

      if found <- setting(settings, "host")
        stdout.println(`host: ${found.get()}`)

      region <- setting(settings, "region").getOrDefault("eu-west-1")
      stdout.println(`region: ${region}`)

      for key in ["port", "retries", "backlog"]
        if raw <- setting(settings, key)
          parsed <- parsePort(raw.get())
          if parsed.isOk()
            stdout.println(`${key}: ok ${parsed.ok()}`)
          if parsed.isError()
            stdout.println(`${key}: error ${parsed.error()}`)
        else
          stdout.println(`${key}: not configured`)

      parsePort("8080").whenOk(showPort)
      parsePort("8080").whenError(showProblem)
      parsePort("80").whenOk(showPort)
      parsePort("80").whenError(showProblem)

It prints:

host: example.org
region: eu-west-1
port: error '8o80' is not a whole number
retries: error 3 is outside 1024 ... 65535
backlog: not configured
listening on 8080
cannot listen: 80 is outside 1024 ... 65535

setting returns an Optional of String, empty when the key is missing; a Dict has no get() of its own, so the Optional is built from contains and getOrDefault. parsePort returns a Result carrying either the port or the reason it was refused. (The ? on its return type declares a slot that holds no object until the body assigns one; the compiler proves every path assigns it.) The program then reads these in four different ways, in this order: a guard (if found <- setting(...)) that enters its block only when the Optional is set; getOrDefault, which never looks inside; a pair of matched checks, isOk() then ok() and isError() then error(); and callbacks, where whenOk and whenError are handed a plain function and only the side that is set calls it. A callback that is not pure is an Acceptor and one that is pure is a Consumer; both overloads exist, and Functions as Values: the Type Hierarchy explains why.

Most reads need no check at all. getOrDefault and the callbacks are the ways to take a value out without writing one, and you will use them more than you expect.

The rule

A read is legal where a check in the same block has proved it; this book calls such a read a safe access. These are the pairs:

The check	What it proves
`x?`, or a guard `if x <- f()`, on an `Optional`	`x.get()`
`r?` on a `Result` (a set `Result` is an ok one)	`r.ok()`
`r.isOk()`	`r.ok()`
`r.isError()`	`r.error()`
`it.hasNext()` on an `Iterator`	`it.next()`

The Iterator is the third safe container: the compiler treats Optional, Result and Iterator the same way, so next() without a hasNext() check is the same error as a bare get().

Take the check away and the compiler refuses the read:

maybe <- nickname("Stephen")
stdout.println(maybe.get())

EK9     : Error   : E08030: 'get' on line 17 position 27: 'maybe as Optional of String': has not been checked before access
             See: ek9 -h E08030

Match the wrong check to the accessor and you get the same code. The compiler does not distinguish a missing check from the wrong one; both are "not checked":

if parsed.isError()
  stdout.println(`age ${parsed.ok()}`)

EK9     : Error   : E08030: 'ok' on line 21 position 37: 'parsed as Result of (Integer, String)': has not been checked before access
             See: ek9 -h E08030

And the same rule holds for the iterator:

cursor <- colours.iterator()
stdout.println(cursor.next())

EK9     : Error   : E08030: 'next' on line 10 position 28: 'cursor as Iterator of String': has not been checked before access
             See: ek9 -h E08030

The legal forms are a loop and an if, each with its hasNext() in the head:

#!ek9
defines module book.iteration

  defines program

    Iteration()
      stdout <- Stdout()
      colours <- ["red", "green", "blue"]
      cursor <- colours.iterator()

      while cursor.hasNext()
        stdout.println(cursor.next())

      noColours <- List() of String
      emptyCursor <- noColours.iterator()
      if emptyCursor.hasNext()
        stdout.println(emptyCursor.next())
      else
        stdout.println("nothing to read")

red
green
blue
nothing to read

Where a Rust or Java reader goes wrong first

The shape every Rust or Java reader writes first is the else:

if parsed.isOk()
  stdout.println(`age ${parsed.ok()}`)
else
  stdout.println(`problem ${parsed.error()}`)

EK9     : Error   : E08030: 'error' on line 24 position 41: 'parsed as Result of (Integer, String)': has not been checked before access
             See: ek9 -h E08030

It does not compile, and the reason is the cost this chapter owns. In Rust, Result has two states and match has two arms; "not Ok" is "Err". An EK9 Result has four states, because a Result that carries a value and an error is a legal, set value:

#!ek9
defines module book.resultstates

  defines function

    orUnset() as pure
      -> text as String
      <- rtn as String: text? <- text else "(unset String)"

  defines program

    FourStates()
      stdout <- Stdout()

      neither <- Result() of (Integer, String)
      okOnly <- Result(42, String())
      errorOnly <- Result(Integer(), "no answer")
      both <- Result(42, "answered, with a caveat")

      for outcome in [neither, okOnly, errorOnly, both]
        stdout.println(`set ${outcome?} ok ${outcome.isOk()} error ${outcome.isError()} printed ${orUnset($outcome)}`)

      nothing <- Optional() of String
      something <- Optional("present")
      stdout.println(`empty ${orUnset($nothing)} full ${orUnset($something)} set ${nothing?} ${something?}`)

      stdout.println("before")
      stdout.println($neither)
      stdout.println(`[${neither}]`)
      stdout.println("after")

set false ok false error false printed (unset String)
set true ok true error false printed {42}
set false ok false error true printed {no answer}
set true ok true error true printed {42, answered, with a caveat}
empty (unset String) full present set false true
before
after

The first four lines are the four states: neither, ok only, error only, both. So "not ok" does not tell you there is an error, and the else of isOk() proves nothing about error(). Check each side you intend to read, with its own if, as the settings program does. The price is two if statements where Rust has one match. What it buys is that "answered, with a caveat" is a value you can express and cannot silently mishandle.

Notice also what is missing from that output. $ of an empty Optional, or of a Result with neither side set, is an unset String; and printing an unset String, or interpolating one into a line, prints nothing at all, not even the newline. The program asked for two lines between before and after, and neither appeared. That is how the tri-state model behaves today, and it is a genuine cost: a debugging println of an empty container tells you nothing. Show emptiness through a helper, as orUnset does here, or print the ? result.

What keeps the proof

The proof lives in the block that checked. Within it, quite a lot is allowed:

#!ek9
defines module book.safeshapes

  defines constant
    LIMIT <- 3

  defines function

    attempts() as pure
      <- rtn as Optional of Integer: Optional(5)

    answer() as pure
      <- rtn as Result of (Integer, String): Result(7, String())

    flag() as pure
      <- rtn as Boolean: true

    note()
      -> text as String
      Stdout().println(text)

    describe() as pure
      -> candidate as Integer
      <- rtn as String: `attempt ${candidate}`

  defines program

    SafeShapes()
      stdout <- Stdout()
      tries <- attempts()
      outcome <- answer()
      first <- flag()
      second <- flag()

      if tries? and tries.get() > LIMIT
        stdout.println(`and: ${tries.get()}`)

      if tries?
        if tries.get() > LIMIT
          stdout.println(`nested: ${tries.get()}`)

      if tries?
        note("a call between the check and the read")
        stdout.println(`after the call: ${tries.get()}`)

      if (tries? and tries.get() > 1) and describe(tries.get())?
        stdout.println(describe(tries.get()))

      if (first or second) and tries?
        stdout.println(`grouped or: ${tries.get()}`)

      if outcome?
        stdout.println(`a set Result proves ok: ${outcome.ok()}`)

and: 5
nested: 5
a call between the check and the read
after the call: 5
attempt 5
grouped or: 5
a set Result proves ok: 7

In order, the proof survives a read in the right-hand conjunct of an and; a nested if, which is still inside the safe-access block; a call between the check and the read; a parenthesised and group, with the read even inside a call argument; and a parenthesised or ANDed with the check, because the whole group is one conjunct. A set Result proves ok() too.

What ends it

or proves nothing. If either side can be true without the check, the body can run with the value unset:

if tries? or retry
  stdout.println(`read: ${tries.get()}`)

EK9     : Error   : E08030: 'get' on line 20 position 38: 'tries as Optional of Integer': has not been checked before access
             See: ek9 -h E08030

Precedence matters here, as it does in C and Python: and binds tighter than or. (first or second) and tries? keeps the proof, as the program above showed. Written without the parentheses, first or second and tries? means first or (second and tries?), and the proof is gone:

if first or second and tries?
  stdout.println(`ungrouped: ${tries.get()}`)

EK9     : Error   : E08030: 'get' on line 21 position 43: 'tries as Optional of Integer': has not been checked before access
             See: ek9 -h E08030

The other thing that ends the proof is a store or a mutation of the checked variable inside the block. Volume 1 showed the plain assignment. The rule is wider than that, because the proof was a fact about the old object:

if host?
  host := fallback()
  stdout.println(`connecting to ${host.get()}`)

The assignment is a re-point, and so is the head of a nested if that stores before it tests:

if host := fallback() then host?
  stdout.println(`inner: ${host.get()}`)
stdout.println(`outer: ${host.get()}`)

EK9     : Error   : E08040: ':=' on line 16 position 13: 'host as Optional of String': Reassignment/mutation within possible 'safe method access' scope is not allowed
             See: ek9 -h E08040
EK9     : Error   : E08040: ':=' on line 23 position 16: 'host as Optional of String': Reassignment/mutation within possible 'safe method access' scope is not allowed
             See: ek9 -h E08040

Mutation counts, not only assignment. :=: empties the object in place; the variable still points at it, and the compiler treats that as the same offence:

name <- Optional("Steve")
if name?
  name :=: Optional() of String
  stdout.println(name.get())

EK9     : Error   : E08040: ':=:' on line 10 position 13: 'name as Optional of String': Reassignment/mutation within possible 'safe method access' scope is not allowed
             See: ek9 -h E08040

The proof follows the object, not the text of the block. A call that cannot reach the checked object keeps it - the note(...) call in the third shape above. A call that may change the object ends it, whether the object is handed over or reachable some other way. The compiler does not read the callee to decide; it reads the callee's signature. forget is not pure, so the compiler must assume it can empty what it is given - and it does:

forget()
  -> target as Optional of String
  target :=: Optional() of String

So a read after calling it needs a new check:

name <- Optional("Steve")
if name?
  forget(name)
  stdout.println(name.get())

EK9     : Error   : E08030: 'get' on line 17 position 28: 'name as Optional of String': has not been checked before access
             See: ek9 -h E08030

In Java this compiles and throws NoSuchElementException at run time; here it never runs. Declare the callee as pure and the proof survives the call, because a pure function cannot mutate what it is given - the :=: in forget would then be E08120. That is the bargain: pure is a promise the compiler checks, and the flow analysis spends it. The cost falls on a non-pure function that only reads the object; the compiler cannot tell it from forget, so it ends the proof too. Operators and Pragmatic Purity, in Full is the rest of that story.

You may not ignore one either

The other half of the discipline is that you cannot drop the answer. A function that returns a Result or an Optional and is called as a bare statement is an error:

save("record")
lookup("missing")

EK9     : Error   : E11052: 'save' on line 19 position 6: method returns Result or Optional but result is discarded - must always be checked
             See: ek9 -h E11052
EK9     : Error   : E11052: 'lookup' on line 20 position 6: method returns Result or Optional but result is discarded - must always be checked
             See: ek9 -h E11052

This is Go's ignorable err closed off. The failure the function reported cannot be lost between its return and your next line. It is E11052, and it takes precedence over the plainer "pure return discarded" rule for these two types.

The cost

You cannot stash a get() behind an arbitrary Boolean and carry it around. The analysis is scoped and syntactic: it recognises a specific set of checks in specific positions and nothing cleverer. Keep the answer in a variable, or ask a helper function (present returns candidate?), and the compiler sees no check at the read:

ready <- tries?

if ready
  stdout.println(`stashed: ${tries.get()}`)

if present(tries)
  stdout.println(`helper: ${tries.get()}`)

EK9     : Error   : E08030: 'get' on line 21 position 41: 'tries as Optional of Integer': has not been checked before access
             See: ek9 -h E08030
EK9     : Error   : E08030: 'get' on line 24 position 40: 'tries as Optional of Integer': has not been checked before access
             See: ek9 -h E08030

There is no unwrap to reach for when you are sure, and you are not allowed to be sure. A Result takes two if statements where Rust takes one match.

That rigidity is the point, not an accident. Every read of an Optional or Result has its proof within a screenful of it, in the same block; a reviewer never has to ask whether a check three functions away still holds. And the analysis can stay that simple because it is that strict. How the languages compare:

Language	The form that is legal anywhere	When you are wrong
Java	`Optional.get()`	`NoSuchElementException`, at run time
Rust	`Option::unwrap()`, `Result::unwrap()`	a panic, at run time
Kotlin	`value!!`	`NullPointerException`, at run time
Go	ignoring the returned `err`	nothing, until much later
EK9	none	E08030, E08040 or E11052, at compile time

The rows are not equally bad; Rust's match is exhaustive and honest, and Kotlin's ?. and ?: are close to EK9's guard and getOrDefault. The difference is that in each of these languages the unsafe form exists, and a deadline will reach for it. In EK9 it does not.

Where this stops

EK9 has no null keyword and no way to write one (E01073). An Optional is a real object even when it is empty, so it is never "null"; it is unset. A slot declared with a ? type and not yet assigned is the case that remains, and the compiler's job is to prove such a slot is never read. That is what the checks in this chapter are part of. EK9 is designed so that absence is unobservable; the compiler checks every read; the proof is in progress. Every read position in the grammar is classified and checked today; the layers that would turn that into a theorem are still being built, and this book says designed so that until they are.² What the chapter has shown you is what the compiler enforces now, with the code it prints.

The API is ek9 -h Optional and ek9 -h Result. whenPresent, whenOk and whenError take an Acceptor or a pure Consumer. ↩
Hoare's own account of the null reference, and the sources behind the rules this chapter keeps, are in The Evidence, the appendix. ↩

In the EK9 reference: EK9 Standard Types