EK9 in Depth

Dynamic Functions and Classes

Streams and Pipelines in Volume 1 left a footnote: there is a shorter way to write a one-off pipeline function where you use it, without giving it a name, called a dynamic function, and the second volume covers it. Constructs at a Glance left another: a function holds no state, except a dynamic function with a capture list, whose captured values become its fields. This chapter pays both. If you know closures from JavaScript, lambdas from Java or Kotlin, or delegates from C#, you already know what a dynamic function is for. The difference is in what it can see. A JavaScript closure sees every variable in scope, whether you meant it to or not; a Java lambda sees every effectively-final one. An EK9 dynamic function sees only what you listed, and the compiler turns that list into the function's fields.

A function that carries state

A rule that gives a discount above a threshold has two settings, the threshold and the rate. Written as a plain function it would need both as parameters on every call, which defeats the point of handing the rule to something else. Written as a dynamic function, the settings travel inside it:

#!ek9
defines module book.vol2.dynamic.discount

  defines function

    discountRule() as pure abstract
      -> amount as Float
      <- rtn as Float?

    thresholdRule() as pure
      ->
        threshold as Float
        rate as Float
      <-
        rule as discountRule: (threshold, rate) is discountRule as pure function
          rtn: (amount >= threshold) <- amount * rate : 0.0

  defines program

    Discounts()
      stdout <- Stdout()

      bulk <- thresholdRule(threshold: 100.0, rate: 0.10)
      loyalty <- thresholdRule(threshold: 20.0, rate: 0.02)

      amounts <- [15.0, 60.0, 250.0]
      for amount in amounts
        stdout.println(`${amount}: bulk ${bulk(amount)} loyalty ${loyalty(amount)}`)

      stdout.println("bulk discounts by pipeline:")
      cat amounts | map with bulk > stdout

It prints:

15.0: bulk 0.0 loyalty 0.0
60.0: bulk 0.0 loyalty 1.2
250.0: bulk 25.0 loyalty 5.0
bulk discounts by pipeline:
0.0
0.0
25.0

discountRule is an abstract function: a signature with no body, and a function type of its own. Functions as Values: the Type Hierarchy covers how EK9 gives every function a type from its shape; here it is enough that a dynamic function must implement something abstract (or a function declared as open), and that the abstract function supplies the parameters and the return variable the body works with. The whole dynamic function is one expression:

(threshold, rate) is discountRule as pure function
  rtn: (amount >= threshold) <- amount * rate : 0.0

The parenthesised list is the capture list. threshold and rate are copied in from the enclosing scope (the names are copied, not the objects; What a capture is, below, has the precise rule), and from then on this function's names for them are its own. Of the enclosing scope's variables the body sees only the captures; beyond those it has the abstract function's parameter (amount) and its return variable (rtn). That last point has one trap. Inside thresholdRule the dynamic function's return variable is the abstract function's rtn, so thresholdRule's own return must be called something else. Here it is rule.

The two rules built from the one factory behave differently because each carries its own threshold and rate. Neither rule's settings can be reached from outside. A rule is also an ordinary value, so map with bulk takes it as a pipeline stage exactly as it would take a named function. That is the point of Streams and Pipelines: every stage is a function value, and a function value can carry state.

A dynamic function can also be written at the place it is used, with no name and no variable. This is the shorter way Volume 1 promised:

#!ek9
defines module book.vol2.dynamic.inline

  defines function

    overLimit() as pure abstract
      -> reading as Integer
      <- rtn as Boolean?

  defines program

    InlineStage()
      stdout <- Stdout()
      readings <- [12, 48, 7, 63, 31]
      limit <- 30

      alarms <- cat readings
        | filter by (limit) is overLimit as pure function (rtn: reading > limit)
        | collect as List of Integer
      stdout.println(`alarms: ${alarms}`)

alarms: [48, 63, 31]

The stage filter by (limit) is overLimit as pure function (rtn: reading > limit) captures limit and gives the function its body as a single rtn: expression in parentheses. Use it when the stage is used once and reads naturally on the line. When the line starts to wrap, give the function a name and a declaration of its own.

What a capture is

The word "capture" means something different in each of the languages you may be thinking of, so it is worth being exact. The experiment is two captures, an Integer and a List, each changed after the dynamic function was made:

#!ek9
defines module book.vol2.dynamic.capturesemantics

  defines function

    counter() as abstract
      <- rtn as Integer?

    sizer() as abstract
      <- rtn as Integer?

  defines program

    CaptureSemantics()
      stdout <- Stdout()

      n <- 1
      f <- (n) is counter as function
        rtn: n
      stdout.println(`f() at capture: ${f()}`)

      n++
      stdout.println(`after n++ (n is ${n}): f() is ${f()}`)

      n := 5
      stdout.println(`after n := 5 (n is ${n}): f() is ${f()}`)

      items <- [10, 20]
      g <- (items) is sizer as function
        rtn: length items
      stdout.println(`g() at capture: ${g()}`)

      items += 30
      stdout.println(`after items += 30 (${items}): g() is ${g()}`)

      items := [7, 8, 9, 11]
      stdout.println(`after items := [...] (${items}): g() is ${g()}`)

f() at capture: 1
after n++ (n is 2): f() is 2
after n := 5 (n is 5): f() is 2
g() at capture: 2
after items += 30 ([10, 20, 30]): g() is 3
after items := [...] ([7, 8, 9, 11]): g() is 3

A capture is a copy of the reference. n++ and items += 30 change the object the variable points at, so the dynamic function, which points at the same object, sees the change. n := 5 and items := [...] point the outer variable at a new object; the dynamic function keeps the old one and does not see it. So the function is not a live window on the variable, and it is not a snapshot of the value either:

If you reach for "capture by value" or "capture by reference" to describe EK9's, neither is right. The working rule is one sentence: the dynamic function and the enclosing scope share the object, and each keeps its own name for it. This matters most with mutable objects such as collections, which is where a captured List can be changed from either side.

The list is the whole interface

Because the capture list is explicit, a body cannot use an outer variable it did not name. Leave limit off the list and the body has nothing to read:

limit <- 10

supplier <- () is limitSupplier as function
  rtn: limit

EK9     : Error   : E50001: 'limit' on line 16 position 13: 'limit': not resolved
             See: ek9 -h E50001
EK9     : Error   : E08090: 'limit' on line 13 position 6: 'limit as Integer': is not referenced anywhere, or not referenced after assignment
             See: ek9 -h E08090

The compiler does not report a missing capture, because there is no such thing to report: the name is not in scope inside the body, so it is simply not resolved (E50001), and the outer limit then reports that nothing read it (E08090). The bug class this closes is the one a JavaScript programmer knows as the loop variable that every callback shares, which var made famous and which Go changed its loop-variable scoping in version 1.22 to stop causing. You cannot capture something by accident in EK9; you can only fail to capture something you meant to.

The other side is also checked. A capture the body never reads is E11018:

rule <- (rate, label) is discountRule as pure function
  rtn: amount * rate

EK9     : Error   : E11018: 'label' on line 17 position 21: unused captured variable in dynamic function or class
             See: ek9 -h E11018

The two checks together make the list exact: everything in it is used, and everything used from outside is in it.

Dynamic classes

A dynamic function implements one abstract function. A dynamic class implements a trait, and may take several. Its captures become private fields, and by can route a trait's methods to a captured object, the way Kotlin's by delegates an interface to a property. Here an auditor timestamps and logs every event, built from two collaborators it does not inherit from:

#!ek9
defines module book.vol2.dynamic.delegation

  defines trait

    Logger
      log() as abstract
        -> message as String

    TimeSource
      now() as pure abstract
        <- rtn as String?

    Auditor
      record() as abstract
        -> event as String

  defines class

    ConsoleLogger with trait of Logger
      override log()
        -> message as String
        Stdout().println(`LOG ${message}`)

    FixedClock with trait of TimeSource
      stamp <- String()

      FixedClock() as pure
        -> stamp as String
        this.stamp :=? String(stamp)

      override now() as pure
        <- rtn as String: stamp

      default operator ?

  defines program

    Delegation()
      stdout <- Stdout()
      logger <- ConsoleLogger()
      clock <- FixedClock("09:00")
      prefix <- "audit"

      auditor <- (prefix, logger, clock) with trait of Auditor, Logger by logger, TimeSource by clock as class
        override record()
          -> event as String
          log(`${prefix} ${now()} ${event}`)

        default operator ?

      auditor.record("login")
      auditor.record("logout")
      stdout.println(`the auditor is also a TimeSource: ${auditor.now()}`)

LOG audit 09:00 login
LOG audit 09:00 logout
the auditor is also a TimeSource: 09:00

(prefix, logger, clock) makes three private fields. Logger by logger and TimeSource by clock say that those two traits are answered by those two fields. The class writes only record(), which calls log() and now() as though it had always had them, and it ends up usable as a TimeSource too, as the last line shows.¹

The class must say when it is set. A capture is a field, and an aggregate with fields needs operator ?; the compiler's own message is plain about the cause:

tag <- (who) with trait of Named as class
  override name() as pure
    <- rtn as String: who

EK9     : Error   : E07236: '(' on line 16 position 13: dynamic class defined on line 16 has 1 captured variable(s) that become private fields but no 'default operator ?' defined: dynamic class with captured variables requires 'default operator ?' to define set/unset semantics for captured fields
             See: ek9 -h E07236

That message is the definition of a capture in the compiler's own words: the captured values become private fields, and a dynamic function or class that has them is an aggregate. default operator ? is true when every capture is set. Write override operator ? as pure when only some captures matter.

Composable by name

Traits and Composition ended on a promise: Scala has a detailed scheme for stacking traits in order, and EK9 never chooses for you. This is that rule at work. EK9 has no linearised super chain, so there is nothing to stack automatically, and the phrase "stackable traits" would mislead a Scala reader. What EK9 has is composition by name.

When two traits give the same method, a class that takes both must say which one wins, or how they combine:

BoldItalic with trait of Bold, Italic

EK9     : Error   : E06150: 'BoldItalic' on line 18 position 4: method 'public String <- render(text as String)' on 'Italic' and 'Bold' : conflicting methods to be resolved
             See: ek9 -h E06150
EK9     : Error   : E06140: 'render' on line 24 position 34: In relation to type 'BoldItalic', and 'render(String)' resolved: public String <- render(text as String) line 7 , public String <- render(text as String) line 12 ambiguous match
             See: ek9 -h E06140
           forAI: methodResolution

E06150 is the clash; E06140 is the same fault met at the call site. The fix is an override that calls each trait by name, and the order is whatever the override writes:

#!ek9
defines module book.vol2.dynamic.composed

  defines trait

    Bold
      render() as pure
        -> text as String
        <- rtn as String: `*${text}*`

    Italic
      render() as pure
        -> text as String
        <- rtn as String: `_${text}_`

    Underlined with trait of Bold
      override render() as pure
        -> text as String
        <- rtn as String: `__${Bold.render(text)}__`

  defines class

    BoldItalic with trait of Bold, Italic
      override render() as pure
        -> text as String
        <- rtn as String: Italic.render(Bold.render(text))

    ItalicBold with trait of Bold, Italic
      override render() as pure
        -> text as String
        <- rtn as String: Bold.render(Italic.render(text))

    Fancy with trait of Underlined

  defines program

    Composing()
      stdout <- Stdout()
      word <- "ek9"
      stdout.println(BoldItalic().render(word))
      stdout.println(ItalicBold().render(word))
      stdout.println(Fancy().render(word))

_*ek9*_
*_ek9_*
__*ek9*__

The first two lines are the same two traits in opposite orders, each written out in a class of its own. The third is a trait laid over a trait: Underlined with trait of Bold wraps Bold.render(text), and Fancy takes that without an override because only one implementation reaches it. Delegation does not settle a clash: a trait taken with by is one more provider of its methods, so two delegated traits with the same method need the same override.

The cost

Explicit capture is more to write than an implicit closure, and you will feel it. A JavaScript arrow function that uses four outer variables costs nothing to write and a capture list costs four names; leave one out and the compiler stops you, leave one in that you no longer use and it stops you again. Composing by name is the same trade on the other axis: to have bold then italic and italic then bold you write two bodies, where Scala lets the order of the trait list decide and you write nothing. With three traits that clash on one method there are as many orders to write as you want to offer, and no default to fall back on. What this buys is that every dynamic function and every composed class can be read from its own text: the names it uses are in its list, the order it applies is in its body, and nothing is decided by a rule you have to remember. Capture is also what a stream stage written inline, and a network handler (Networking: TCP and UDP), are built from.