Operators and Pragmatic Purity, in Full

Using Operators in Volume 1 ended on a promise: an operator is either mutating or non-mutating, EK9 has an exact, enforced model of which is which, and that model meets pure. If you have written C++ or Python you have overloaded operators to mean whatever you liked: << to print, + to concatenate, / to join paths. EK9 keeps the spelling and removes the freedom. The set of operators is closed, every signature is fixed, and the compiler checks which of them change their operand. This chapter is that model, the purity rules built on it, and the three operators that copy, merge and replace. It ends on the guard head that Volume 1 left open.

A closed set with one split

You cannot invent an operator in EK9, and you cannot give one a signature of your choosing. ? returns a Boolean and takes nothing; $ returns a String; <=> takes one argument of its own type and returns an Integer. The compiler's own list of what exists is short enough to hold in your head,¹ and one line of it does the work of this chapter. Every operator either mutates its receiver or does not:

Kind	Operators	`as pure`?
Mutating: changes the object, returns nothing	`:=:` `:~:` `:^:` `+=` `-=` `*=` `/=` `++` `--`	must not be (E07510)
Non-mutating: computes, changes nothing	`+` `-` `*` `/`, the comparisons and `<=>`, `?`, `$`, `$$`, `#?`, `contains`, and the others the list names	must be (E07500)

The split is not a convention, because the compiler enforces it in both directions. Here is a Credit record that does it correctly: + builds a new value and leaves both operands alone, so it is pure; += changes deposit in place, so it is not.

#!ek9
defines module book.operatorsplit

  defines record

    Credit
      pence <- Integer()

      default Credit() as pure

      Credit() as pure
        -> amount as Integer
        pence :=? Integer(amount)

      operator + as pure
        -> other as Credit
        <- rtn as Credit: Credit(pence + other.pence)

      operator +=
        -> other as Credit
        pence += other.pence

      operator $ as pure
        <- rtn as String: `${pence}p`

      default operator ?

  defines program

    OperatorSplit()
      stdout <- Stdout()

      deposit <- Credit(150)
      bonus <- Credit(250)

      sum <- deposit + bonus
      stdout.println(`sum ${sum}, deposit still ${deposit}`)

      deposit += bonus
      stdout.println(`after +=, deposit ${deposit}`)

sum 400p, deposit still 150p
after +=, deposit 400p

Now break each half of the rule. A + that is not marked pure:

operator +
  -> other as Credit
  <- rtn as Credit: Credit(pence + other.pence)

EK9     : Error   : E07500: '+' on line 15 position 15: operator semantics: operator must be declared pure
             See: ek9 -h E07500

And a += that is:

operator += as pure
  -> other as Credit
  pence += other.pence

EK9     : Error   : E07510: '+=' on line 15 position 15: operator semantics: operator must not be declared pure
             See: ek9 -h E07510

Both come from the same early pass over the declarations, before any body is read, so a file with both mistakes and a purity error in a body shows these two first. The compiler is not inspecting what the operator does; it is holding the declaration to what the operator's name already says it does. In C++ and Python nothing in the declaration of + says it must leave its operands alone. Here a reader who sees + knows, from the signature alone, that it returns a new value and that nothing it was given has changed.

What `pure` enforces

pure applies to a function, a method, a constructor or an operator, and it is enforced, transitively, with three rules. A pure body may not reassign anything but its own return variable (E08100). It may not mutate a variable, whether a local, something it was handed, or the return variable itself (E08120). And it may call only code that is itself pure (E08130). Coming from C++ const the shape is familiar, except that there is no const_cast. Coming from Haskell it is weaker: a pure EK9 function may print.

Each rule has a one-line way to break it. The first is the call:

label()
  -> amount as Float
  <- rtn as String: `GBP ${amount}`

receipt() as pure
  -> amount as Float
  <- rtn as String: `Total ${label(amount)}`

EK9     : Error   : E08130: 'label' on line 12 position 33: 'label(amount) => String <- label(amount as Float)': is not marked 'pure', but call is made in a scope that is marked as 'pure'
             See: ek9 -h E08130
           forAI: purityRules

The second is a reassignment of a local. Volume 1 showed the plain case; the message also tells you what is allowed, :=?, which assigns only when the target is still unset:

runningTotal() as pure
  -> prices as List of Float
  <- rtn as Float: 0.0

  running <- 0.0
  for price in prices
    running := running + price
  rtn: running

EK9     : Error   : E08100: ':=' on line 20 position 16: 'running as Float': reassignment not allowed when scope is marked as 'pure' (':=?' is supported)
             See: ek9 -h E08100
           forAI: purityRules

The third is mutation, and it has two edges worth knowing. The return variable may be re-assigned (rtn: running is accepted above, as is the rtn: in every pure function in this chapter) but it may not be mutated; and a parameter is as protected as anything else:

runningSum() as pure
  -> prices as List of Float
  <- rtn as Float: 0.0

  for price in prices
    rtn += price

addTo() as pure
  ->
    prices as List of Float
    extra as Float

  prices += extra

EK9     : Error   : E08120: '+=' on line 28 position 12: mutating variables is not allowed when scope is marked as 'pure'
             See: ek9 -h E08120
EK9     : Error   : E08120: '+=' on line 35 position 13: mutating variables is not allowed when scope is marked as 'pure'
             See: ek9 -h E08120

The first of those two is the one people do not expect. Accumulating into the return variable is the natural loop in every other language, and here it has to be a stream, a for that hands back a value, or a local built up with <- and returned once. Volume 1's Guards and Safe Code shows those shapes; they exist because pure code needs them.

The fix for the first error is one word. Mark label pure and the caller compiles. The program below also shows the two things the rules do not say. A pure function may print (Stdout().println is itself declared pure), and a function that is not pure may call a pure one; the restriction runs one way only.

#!ek9
defines module book.purity

  defines function

    label() as pure
      -> amount as Float
      <- rtn as String: `GBP ${amount}`

    receipt() as pure
      -> amount as Float
      <- rtn as String: `Total ${label(amount)}`

    announce() as pure
      -> text as String
      Stdout().println(text)

    audit()
      -> text as String
      announce(`audit: ${text}`)

  defines program

    PurityRules()
      stdout <- Stdout()
      stdout.println(receipt(12.5))
      announce("a pure function may print")
      audit("a function that is not pure may call a pure one")

      first <- GUID()
      second <- GUID()
      stdout.println(`two calls to a pure constructor, same answer: ${first == second}`)

Total GBP 12.5
a pure function may print
audit: a function that is not pure may call a pure one
two calls to a pure constructor, same answer: false

The last line is there for the reader who assumes pure means referentially transparent. It does not. GUID() is a pure constructor and no two calls agree. What pure promises is about change: nothing is reassigned, nothing handed in is mutated, and nothing impure is called. It does not promise that the same input gives the same output, and it does not promise silence. If your idea of purity is Haskell's, EK9's is a weaker, more practical property, and the three rules above are the whole of what the compiler checks.

Constructors have one more rule, and it explains why every constructor of Credit above says as pure: if any constructor of a type is pure, all of them must be (E05190).

default Credit() as pure

Credit()
  -> amount as Integer
  pence :=? Integer(amount)

EK9     : Error   : E05190: 'Credit' on line 11 position 6: if any constructor is marked pure, all constructors must be pure
             See: ek9 -h E05190
           forAI: purityRules

Purity belongs to the operation

There is no immutable List in EK9, no final and no const. Every type stays mutable; what pure controls is whether this piece of code may mutate it. A List handed to a pure function is the same List the caller holds, and the function is refused only when it writes.

A design note. EK9 moves immutability from the data structure to the processing. A pure function cannot change what it is given, but what it hands back is an ordinary value the caller may go on changing - reasonable mutation of a returned value is the very nature of many function and method calls. That is deliberately less strict than Haskell and the other functional languages. It also keeps Liskov intact: a Java List.of(...) is a List whose add throws, so every caller of List has to wonder which kind it holds, and The Payoffs: Liskov and Hoare comes back to that. And there is no converting back and forth between a mutable and an immutable version of the same type. - Steve

That is also why the compiler can use purity without looking inside a callee. Optional and Result, in Depth ended on a forget function that empties an Optional it was given, and said that declaring it as pure would make the :=: inside it an E08120. Here is that claim, run:

forget() as pure
  -> target as Optional of String
  target :=: Optional() of String

EK9     : Error   : E08120: ':=:' on line 39 position 13: mutating variables is not allowed when scope is marked as 'pure'
             See: ek9 -h E08120

So a pure signature is a promise the compiler has already checked inside the body, and a call to a function the compiler can see is pure does not end the check in Optional and Result, in Depth. A callee that is not pure may do anything, so the analysis assumes the worst. Functions as Values: the Type Hierarchy shows the same signature reading from the other side: the type a function gets depends on whether it is pure.

Copy, merge and replace

Three mutating operators move one object's state into another. In Java you would write clone(), a merge helper and a setter loop; in EK9 they are operators, they mutate the left-hand side, and they return nothing.

target :=: source copies: the target takes the source's state, set or not.
target :~: source merges: the target takes only what it does not already have. A set value is never overwritten.
target :^: source replaces: the target takes what the source has set, overwriting what it holds.

The built-in types come with their own. For a type of your own, default operator :=: generates a copy from the fields, and the other two are always yours to write: asking for the merge or the replace to be defaulted is E07220.² A configuration record is the natural home for them, because the three operators are three different answers to "how do these two settings combine":

#!ek9
defines module book.copymerge

  defines record

    Profile
      name <- String()
      email <- String()
      theme <- String()

      operator :=:
        -> other as Profile
        name :=: other.name
        email :=: other.email
        theme :=: other.theme

      operator :~:
        -> other as Profile
        name :=? String(other.name)
        email :=? String(other.email)
        theme :=? String(other.theme)

      operator :^:
        -> other as Profile
        if other.name?
          name :=: other.name
        if other.email?
          email :=: other.email
        if other.theme?
          theme :=: other.theme

      operator $ as pure
        <- rtn as String: `${shown(name)} ${shown(email)} ${shown(theme)}`

      override operator ? as pure
        <- rtn as Boolean: name? and email? and theme?

  defines function

    shown() as pure
      -> text as String
      <- rtn as String: text? <- text else "-"

  defines program

    CopyMergeReplace()
      stdout <- Stdout()

      stored <- Profile()
      stored.name: "Steve"
      stored.email: "steve@example.org"
      stored.theme: "dark"

      patch <- Profile()
      patch.email: "limb@example.org"
      patch.theme: "light"

      copied <- Profile()
      copied :=: stored
      stdout.println(`copied   [${copied}]`)

      merged <- Profile()
      merged :=: stored
      merged :~: patch
      stdout.println(`merged   [${merged}] the patch could not overwrite`)

      filled <- Profile()
      filled.name: "Steve"
      filled :~: patch
      stdout.println(`filled   [${filled}] the gaps were filled`)

      replaced <- Profile()
      replaced :=: stored
      replaced :^: patch
      stdout.println(`replaced [${replaced}] what the patch had set won`)

copied   [Steve steve@example.org dark]
merged   [Steve steve@example.org dark] the patch could not overwrite
filled   [Steve limb@example.org light] the gaps were filled
replaced [Steve limb@example.org light] what the patch had set won

Notice that the merge into a full profile changes nothing, and that the merge into a partial one fills only its gaps; the body of :~: is :=? per field, which is the whole rule. :^: on a type of your own means what you write: the operator is "replace" by name, and for your own type the compiler does not decide what replacing means. On a built-in it overwrites, as :=: does. All three are mutating operators, so none of them can be declared pure, and none can appear in a pure body (E08120).

References, and what a copy shares

If you come from Java, your intuition has three cases: primitives copy, String and boxed numbers are immutable, and everything else is shared by reference. EK9 has one case. Every value is an object, and <- from a variable gives the new name to the same object:

#!ek9
defines module book.aliasing

  defines program

    Aliasing()
      stdout <- Stdout()

      count <- 10
      sameObject <- count
      count++
      stdout.println(`count ${count} sameObject ${sameObject}`)

      independent <- Integer(count)
      count++
      stdout.println(`count ${count} independent ${independent}`)

      names <- ["Ann"]
      sameList <- names
      names += "Bo"
      stdout.println(`names ${names} sameList ${sameList}`)

count 11 sameObject 11
count 12 independent 11
names ["Ann", "Bo"] sameList ["Ann", "Bo"]

++ is a mutating operator, so it changes the one Integer both names point at; there is no boxed immutable Integer standing between you and the change. To get an independent value you construct one: Integer(count) is a copy.

The same rule governs :=:, and it is the trap in the generated operator. default operator :=: copies each property from the source, and a property that holds an object now refers to the same object the source's property does. The copy is shallow, exactly as b <- a and passing a parameter are:

#!ek9
defines module book.shallowcopy

  defines record

    Team
      name <- String()
      members <- List() of String

      Team()
        ->
          name as String
          members as List of String

        this.name: name
        this.members: members

      default operator $
      default operator ?
      default operator :=:

  defines program

    ShallowCopy()
      stdout <- Stdout()

      original <- Team("core", ["Ann", "Bo"])
      copy <- Team()
      copy :=: original
      original.members += "Cy"
      stdout.println(`original ${original}`)
      stdout.println(`copy     ${copy}`)

      roster <- ["Di", "Ed"]
      duplicate <- List() of String
      duplicate :=: roster
      roster += "Flo"
      stdout.println(`roster ${roster} duplicate ${duplicate}`)

original Team(name=core, members=["Ann", "Bo", "Cy"])
copy     Team(name=core, members=["Ann", "Bo", "Cy"])
roster ["Di", "Ed", "Flo"] duplicate ["Di", "Ed"]

The copy of the team sees Cy, because both teams hold one List. The last line is the contrast: a List copied with :=: at its own level gets its own container, holding the same elements. A shallow copy of a record is the right answer when the fields are values and a bug when one of them is a mutable collection you meant to own. There is no default deep copy and no clone(); a deep or selective copy is an operator :=: you write, deciding field by field what is duplicated and what is shared. ek9 -q 2100 answers it with an insurance example in which a renewed policy duplicates its drivers and shares its wording.

This is also where the E08040 of Optional and Result, in Depth comes from. A copy, a merge and a replace are all mutations, so each ends a safe-access proof just as += does, and each is refused inside a pure body for the same reason.

The re-point head

"Re-point" is this book's word for making an existing name refer to another object, and it covers both spellings: the := of this head and the : that other chapters use to assign to an existing variable. Neither changes the object the name used to refer to; the name now refers to another.

Volume 1 gave you two assignment forms in a guard head: <- declares a new variable and enters only when it is set, and :=? assigns only when the variable is still unset. The third form handles a variable that already exists and must be given a new value: :=, followed by then and a check.

The part of a control-flow head before then is the pre-flow: the statement that runs once, before the test. In if latest := reading(...) then latest? the pre-flow is the re-point, and latest? is the condition. A pre-flow is blind: it assigns and checks nothing, so the head needs its own condition after it.

#!ek9
defines module book.thenguard

  defines constant
    WARM <- 18

  defines function

    reading() as pure
      ->
        readings as Dict of (String, Integer)
        station as String
      <- rtn as Optional of Integer: Optional() of Integer

      if readings contains station
        rtn: Optional(readings.getOrDefault(station, Integer()))

  defines program

    ThenGuard()
      stdout <- Stdout()
      readings <- {"roof": 21, "cellar": 9}
      latest <- Optional() of Integer

      if latest := reading(readings, "roof") then latest? and (latest.get() > WARM)
        stdout.println(`roof is warm: ${latest.get()}`)

      if latest := reading(readings, "cellar") then latest? and (latest.get() > WARM)
        stdout.println(`cellar is warm: ${latest.get()}`)
      else
        stdout.println("cellar is not warm")

      if latest := reading(readings, "attic") then latest?
        stdout.println(`attic: ${latest.get()}`)
      else
        stdout.println("attic: no reading")

      stdout.println(`after the last check, latest is set: ${latest?}`)

roof is warm: 21
cellar is not warm
attic: no reading
after the last check, latest is set: false

The get() is legal in the body and in the right-hand side of the and for the reason Optional and Result, in Depth gave: the check sits in the same head. Leave the check out and the head has no condition at all:

if latest := reading("roof")
  stdout.println(`roof: ${latest.get()}`)

EK9     : Error   : E07340: 'latest' on line 16 position 9: an assignment pre-flow checks nothing, so this 'if' has no condition - use a guard ('<-', ':=?') or add a condition - 'then x?' to enter only when it is set: without a control, failed to find subject of flow
             See: ek9 -h E07340

The last line of the program is the cost of :=. The head overwrote latest on every pass, and when reading found nothing it overwrote it with an unset Optional, so after the attic lookup the old value is gone. If you want to keep what you have when the new answer is unset, you want :=? or a new name. The further condition in the first two heads is parenthesised, as the compiler's own help writes it.

What it costs

You cannot write a quick logging call, a cache write or a counter increment into a function you have marked pure, and you cannot call a helper that nobody marked. The first time it stops you, three levels down a call chain, it is a genuine nuisance: the fix is to thread as pure up through every helper in between, which is tedious, and where one of them really does mutate, to split the mutation out into a function that is not pure and call it from the outside. There is no unsafe and no const_cast. Constructors are all-or-nothing, an operator's purity is fixed by its name, and a generated copy is shallow whether or not that is what you meant.

What that buys is that a signature is a statement the compiler has already checked. The three rules above are checked inside every pure body, and a call site needs only the callee's signature to know what it may do to the arguments.

That is why a call to a pure function does not end the check in Optional and Result, in Depth, and it is why the function types in Functions as Values: the Type Hierarchy fork on purity: the Predicate and the Assessor have the same shape and differ only in whether the function is pure.

Where this stops

The compiler proves, for every pure operation it accepts, that nothing in the body reassigns but the return variable, mutates a variable, or calls what is not pure; and for every operator, that its purity matches its kind. It does not try to prove that a pure function is deterministic (GUID()), that it terminates, or that a hand-written :=:, :~: or :^: means what its name says; the model fixes the shape of an operator and leaves its meaning to you. A generated :=: is shallow, and the compiler will not warn you that a field you meant to own is shared.

ek9 -h -E4 prints the paradigm summary, which carries the full list in its OPERATORS block; ek9 -h operator is the shorter help. ↩
default operator writes the mechanical operators (<=>, $, $$, #?, :=:, ?) from a type's fields. #^, :~: and :^: are always written by hand, and asking for one by default is E07220. ↩