EK9 in Depth

Organising a Growing Codebase

Components in Volume 1 kept everything in one file and said that "splitting a project across modules" was a topic for the second volume. Your programs are now outgrowing one file, and the first question a Java, Go or Rust programmer asks is where the build file lives. In Java it is a pom.xml or a build.gradle, a second language with its own plugins; in Go it is go.mod and the directory tree; in Rust, Cargo.toml. In EK9 it is a block inside one of your source files, and the compiler is the only build tool. This chapter covers how files make a module, how one module reaches another, where the tests go, and a view about comments that the compiler enforces as far as a compiler can.

One build, many files

Compiling a file compiles exactly that file. A second file is invisible to it, even if both declare the same module, until the file you name carries a defines package block with applyStandardIncludes <- true, which puts every .ek9 file under its directory into the build. Here is a small inventory project, laid out by what each file is about:

catalogue/items.ek9
dev/inventoryTests.ek9
main.ek9
pricing/pricing.ek9

Four files, three folders, two modules. main.ek9 names the module book.inventory, holds the package block and the program; the record Item is in catalogue/items.ek9, which declares the same module; the pricing rules are a different module, book.inventory.pricing, in their own folder. The folder names mean nothing to the compiler; modules are named by defines module, never by a path. Here are the files, starting with the one you name:

defines module book.inventory

  references
    book.inventory.pricing::priceWithTax

  defines function

    stockLine() as pure
      -> item as Item
      <- rtn as String: `${item.name}: ${priceWithTax(item.price)}`

    stockLines() as pure
      -> items as List of Item
      <- rtn as List of String: cat items | map with stockLine | collect as List of String

  defines program

    Inventory()
      stdout <- Stdout()
      stock <- [Item("lamp", 30#GBP), Item("desk", 120#GBP)]
      stdout.println(stockLines(stock))

  defines package
    version <- 1.0.0-0
    description <- "A small inventory: one module over two files, plus a pricing module"
    applyStandardIncludes <- true

The compiler wrote the package block last when it formatted the file; its position is not significant. The other two files declare their own modules and nothing else:

defines module book.inventory

  defines record

    Item
      name <- String()
      price <- Money()

      default Item() as pure

      Item() as pure
        ->
          itemName as String
          itemPrice as Money

        name :=? itemName
        price :=? itemPrice

      default operator ?

defines module book.inventory.pricing

  defines constant
    VAT_RATE <- 0.2

  defines function

    priceWithTax() as pure
      -> net as Money
      <- rtn as Money: net + (net * VAT_RATE)

Run from the project folder, ek9 main.ek9 builds and runs the program, and prints:

["lamp: 36.00#GBP", "desk: 144.00#GBP"]

stockLine uses Item with no references entry, because main.ek9 and items.ek9 are one module; it needs one for priceWithTax, because that is another. A references block is for other modules. It does not add a file to the build: only the package block does.

Leave the package block out and the files compile one at a time. The error you meet first is for the cross-module reference, because it is checked before anything else:

  references
    book.inventory.pricing::priceWithTax

EK9     : Error   : E03030: 'priceWithTax' on line 5 position 28: 'book.inventory.pricing::priceWithTax' reference does not resolve
             See: ek9 -h E03030

The line to read is the one it names, a references entry that points at a module the build does not contain. A symbol from the same module in a file the build does not contain fails later, as E50001, "not resolved". applyStandardIncludes takes every file it finds; the alternative is includeFiles, a list that names each file and silently omits any you forget to add.

A consequence of the build being a block in a source file: the program you run is in the file you name. A file holding only the package block and no program is a legitimate thing to compile and has nothing to run (ek9: no programs to run).

What goes in dev/

Tests live in a dev/ folder beside the named file, and the standard includes leave that folder out of a normal build, so no test code reaches a deployed program. ek9 -t main.ek9 builds the project with dev/ included and runs every @Test program in it. A test is its own module; it names what it tests in a references block, as any other module would.

The package block is needed for exactly one reason: the file you name must reach other source files that are not under dev/. A project of a single source file with a dev/ folder beside it needs no package block at all, and the file you name need not be called main.ek9: ek9 -t on that file picks up every program in the dev/ folder beside it, nested folders included. Add a second source file that the named file uses, and the package block (with applyStandardIncludes) is what puts it in the build.

defines module book.inventory.tests

  references
    book.inventory.pricing::priceWithTax
    book.inventory::Item
    book.inventory::stockLine
    book.inventory::stockLines

  defines program

    @Test
    PriceWithTaxTest()
      assert priceWithTax(100#GBP) == 120#GBP

    @Test
    StockLineTest()
      assert stockLine(Item("lamp", 30#GBP)) == "lamp: 36.00#GBP"

    @Test
    ItemIsSetTest()
      assert Item("lamp", 30#GBP)?

    @Test
    StockLinesTest()
      stock <- [Item("lamp", 30#GBP), Item("desk", 120#GBP)]
      assert length stockLines(stock) == 2

ℹ️ Found 4 tests:
  4 assert (unit tests with assertions)

Executing 4 tests...

✅ PASS book.inventory.tests::ItemIsSetTest [Assert] (2ms)
✅ PASS book.inventory.tests::PriceWithTaxTest [Assert] (3ms)
✅ PASS book.inventory.tests::StockLineTest [Assert] (3ms)
✅ PASS book.inventory.tests::StockLinesTest [Assert] (5ms)

Summary: 4 passed, 0 failed (4 total)
Types: 4 assert
Duration: 15ms

The project passes the coverage bar because the program is kept thin: it builds a list and prints one line, and everything that can be tested is a function. Testing the Compiler Trusts returns to how coverage is counted; Packaging and Sharing Your Program shows what happens to a project that cannot meet it.

How modules reach each other

The references block lists what a module imports, one module::name per line, and it must be alphabetically ordered:

  references
    book.inventory.pricing::priceWithTax
    book.inventory.audit::auditLine

EK9     : Error   : E11026: 'book' on line 6 position 4: 'book.inventory.audit::auditLine' should come before 'book.inventory.pricing::priceWithTax' - references must be alphabetically ordered: references not in alphabetical order - reorder for consistency
             See: ek9 -h E11026
EK9     : Error   : E03030: 'priceWithTax' on line 5 position 28: 'book.inventory.pricing::priceWithTax' reference does not resolve
             See: ek9 -h E03030

The second error is only there because this file is compiled alone, with neither module present. E11026 is one of the compiler's consistency rules rather than a matter of taste, which is the volume's argument in miniature: when ordering is a compile error, a diff of a references block is always small, and nobody argues about it in review.

Two things you would expect a compiler to stop are not stopped. A module reference cycle is legal. These two modules reference each other, compile together, and run:

defines module book.cycle.alpha

  references
    book.cycle.beta::fromBeta

  defines function

    fromAlpha() as pure
      <- rtn as String: "alpha"

    roundTrip() as pure
      <- rtn as String: fromBeta()

  defines program

    Cycle()
      Stdout().println(roundTrip())

  defines package
    version <- 1.0.0-0
    description <- "Two modules that reference each other"
    applyStandardIncludes <- true

defines module book.cycle.beta

  references
    book.cycle.alpha::fromAlpha

  defines function

    fromBeta() as pure
      <- rtn as String: `beta then ${fromAlpha()}`

    betaName() as pure
      <- rtn as String: "beta"

beta then alpha

All the files of a build are compiled together, so there is no ordering for a cycle to violate. EK9's cycle errors are about type hierarchies (E05020), dependency injection (E08190) and dependencies between packages; a cycle of modules is a design smell the compiler will not decide for you. What it does measure is coupling: a module that references more than ten other modules is E11016, which counts modules, not types, and the cure its help text gives is a wrapper module that groups the dependencies.¹

The other is how constants are spread. A module's constants belong in one file, and when they are in three or more files of one module the compiler says so. The project is three files of module book.scattered, each declaring a constant:

EK9     : Error   : E11066: 'book' on line 2 position 15: Constants found in 3 files within module 'book.scattered': constants should be consolidated into a single file
             See: ek9 -h E11066

(The error is reported on the module line of one of the three files; with two files it is allowed, so a module can have its constants beside its main program or in a file of their own.) It is a small rule with a large effect on how fast someone new finds the vocabulary of a module.

Comments

A comment is the one thing in a program the compiler cannot check. Types, absence, purity, coverage and coupling are all verified; a comment can say anything, can go stale the day after it was true, and nothing will ever notice. That puts it outside "if it compiles, it is good", and it is why EK9 takes a position.

A design note. A program is organised by its constructs. An application wires its components and aspects together, a service is a web service, and classes, functions and constants are exactly that; a text construct holds what must be localised, a trait is a conceptual abstraction and a record carries data between parts. Because the construct says what something is and the role it plays, no comment or @ annotation has to. Meaningful naming is meant to do the rest - EK9 refuses a name that says nothing - and the strict semantics of the operators make each one self-explanatory wherever it appears. I have been misled by comments for forty years: a wrong comment is worse than none, because the reader trusts it and the code can no longer correct them. Say what the code does in the code; say why in a comment, and only when the reader could not otherwise know. - Steve

The language does most of this by making the code carry the intent. A literal in a comparison must be named (E11064), and a literal repeated three times must be a constant (E11065), so the comment that would have explained a bare number is replaced by a name the compiler holds you to:

if stock < 5  // reorder when low

becomes if stock < REORDER_LEVEL, as in the primer. A constrained type states a range in a place the compiler enforces, where a comment would only state it. A ? on a type, a pure on a function and an assert in a test are checked claims about the same things that comments make unchecked ones about.

EK9 has four comment forms, and no others:

#!ek9
<?-
  The documentation form, written before the construct it describes.
-?>
defines module book.comments

  defines function

    <?-
      Why this function exists. The compiler checks the lines below; nothing checks this.
    -?>
    describe() as pure
      -> count as Integer
      <- rtn as String: `${count} item(s)` // a line comment runs to the end of the line

  <!--
    A block comment: everything in here is ignored.
  -->

  <!-
    The second block form, also ignored.
  -!>

  defines program

    Comments()
      stdout <- Stdout()
      stdout.println(describe(3))

3 item(s)

// runs to the end of the line, and is the only form Volume 1 used. <!-- --> and <!- -!> are block forms; the first is there for people who think in HTML, and the second is the one EK9's own tooling uses to comment lines out. <?- -?> is the documentation form: the grammar marks it as intended to be attached to the source as documentation, and it is the one to use for a concept or a reason, as the function above does. To the compiler all four are hidden text.

C's /* */ is not one of the four, deliberately. Written in a file it is a parse error, and the error is helpful about it:

    /* A C-style block comment is not one of EK9's four comment forms. */

Syntax  : '/* A C-style block comment is not one of EK9's four comment forms. */' on line 6 position 4: extraneous '/* A C-style block comment is not one of EK9's four comment forms. */' expecting valid statement or expression 
Unable to parse source file [cStyleComment.ek9]

Hint: 'ek9 -ff cStyleComment.ek9' can repair what stops this file parsing; it never renames anything, and rewrites the file only if the result parses. It would make 1 repair:
  line 6: a '/* */' comment written as '<!- -!>' - EK9's block comment

That is a Syntax error, not an E code, because the file never got as far as the checks that have codes. ek9 -ff will make the repair it names.

Use a documentation comment when something about the code cannot be expressed in it: a domain rule, a trade-off, why an odd thing is done because a particular edge case bites. Use a block comment for the rare "this looks strange and is deliberate". Do not use either to narrate a line the reader can already see. The examples in this book follow that rule, which is why they have almost none.

What it costs

The build in a source file has a price. You cannot script it: there is no task runner, no plugin point and no way to say "also run this step". The package block holds metadata, includes and dependencies, and a Maven user will miss the lifecycle. A folder layout that the compiler ignores means that nothing stops you putting a file in the wrong one; the compiler checks modules and references, not your tidiness. And a rule like sorted references or consolidated constants is a rule you did not choose.

What that buys is that every EK9 project builds the same way, and the way is visible in the code it builds: a new reader never opens a second file in a second language to learn how the first is assembled. The compiler's checks (sorted references, consolidated constants, coupling, coverage) reach the layout of the project as they reach its functions. Comments are the exception that proves the rule, and the volume has argued as far as a compiler can: it cannot check the why, so it makes sure you rarely need to write one.