Where to go next

The book has taught you to write EK9. This appendix is about what to reach for once the program exists: finding a fault in a running program, finding where it is slow, writing the tests nobody wants to write, and finding out whether the tests you did write are any good. Every one of these is a command of the ek9 binary today. The examples run against one small project, the postage function from Quality Built In with its six tests, and every block is output from a real run.

The graphical side of all this, the editor, the debugger's windows and the refactoring tools, is not here. It has its own book, The EK9 IDE and Refactoring Toolset, a living web reference; the chapters of this volume stay at the command line.

The debugger and the journal

ek9 -cg file.ek9 compiles with debugging information, and ek9 -d <port> file.ek9 runs the program with a debug agent listening on that port, without waiting for a debugger to attach:

java -Xmx512m -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:4450 -jar './.ek9/jvm/debug/AddressRecord.jar' -r book.quality.address::ShowAddress

That line is what the compiler hands to the launcher for ek9 -d 4450 AddressRecord.ek9. Read it before you use the flag: address=127.0.0.1:4450 means the agent listens on this machine only: a debug port runs whatever code it is sent, so it is never opened to the network. suspend=n means the program starts at once, so the port is for attaching to a program that is already running.

The debugger itself is a library behind three hosts: the IDE, ek9 -dap, which speaks the Debug Adapter Protocol so that an editor of your choice can debug an EK9 program, and ek9 -mcp, which gives an AI assistant the same session as tools. A run can also be journalled, which records every change to one value you choose without stopping the program, then compares runs that passed with runs that failed and writes the difference up as a report. ek9 -h journal is its reference, and it is driven from the IDE or over MCP; the companion book has its workflow.

Profiling

Add a p to any test flag and the run is instrumented. -tp prints a hot-function summary after the coverage summary, -t2p writes it as JSON for a tool to read, and -t6p writes the HTML report with a flame graph. On the six tests:

Profiling: 1 functions profiled
  postage                             Self: 4.108ms  Calls: 6

The figure is the time spent in the function itself, and how many times it was called. Timings differ every run; the count does not. Profiling forces the unoptimised build so that each probe maps to a source line, which is the opposite of how a packaged program is built (-O3), so use it to find which function is hot and not to say how fast the release will be.

Tests nobody wants to write

-fuzztest reads a source file's symbol table and writes @Test programs for the edge values of every function it finds. The output has to go under a dev/ folder, because a test outside one is E81015:

EK9 Test Generation: analyzing postage.ek9
Seed: 7, Max candidates: 200
Found 0 types, 1 functions
Tests will be declared in module book.quality.postage.tests (source module is book.quality.postage) — emitting a references block
Generated 51 test candidates
Survivors: 51 / 51 (100%)
Constants written to: dev/constants.ek9
Output written to: dev/generated.ek9

It wrote 51 candidates, and a header that is worth reading in full. They are scaffolding: the generator wrote the calls and the edge values, and "did NOT decide what the right answer is". Each test asserts only that the call returns a set value, with a comment marking where you must write the expected result from the requirement. The edge values come from the integer literals in the file (the constants of postage give 249, 250 and 251 around the surcharge, for instance). The warning in the header is the one from Testing the Compiler Trusts, stated by the tool: if you keep whatever assertion passes you have frozen today's behaviour, defects included. Keep the useful candidates, set their expectations yourself, and delete the rest.

Do your tests notice?

Coverage and variety measure what ran. Mutation testing asks a different question: if the code were slightly wrong, would a test fail? -fuzzmutate writes one file per single-point change that still compiles, in a directory and a manifest:

EK9 Mutation Testing: analyzing postage.ek9
Seed: 7, Max candidates: 100
Generated 16 mutation candidates
Survivors: 16 / 16 (100%)
Output written to: mutations

"Survivors" here means the mutants compile; whether your tests catch them is the next step. The manifest lists each file, the kind of change, the line and the original and mutated text.

The command-line help also lists a -test flag that is meant to run the kill-check for you; at the time of writing it is accepted and prints nothing further, so the check is a loop of your own. Copy each mutant over the source in a scratch project, run the tests with -t0, and note whether they failed. For the six tests of postage:

mutation_000_boundary_L5.ek9   BOUNDARY  line 5   500 -> 499: survived
mutation_001_boundary_L5.ek9   BOUNDARY  line 5   500 -> 501: survived
mutation_002_boundary_L6.ek9   BOUNDARY  line 6   5000 -> 4999: survived
mutation_003_boundary_L6.ek9   BOUNDARY  line 6   5000 -> 5001: survived
mutation_004_boundary_L7.ek9   BOUNDARY  line 7   300 -> 299: killed
mutation_005_boundary_L7.ek9   BOUNDARY  line 7   300 -> 301: killed
mutation_006_boundary_L8.ek9   BOUNDARY  line 8   450 -> 449: killed
mutation_007_boundary_L8.ek9   BOUNDARY  line 8   450 -> 451: killed
mutation_008_boundary_L9.ek9   BOUNDARY  line 9   900 -> 899: killed
mutation_009_boundary_L9.ek9   BOUNDARY  line 9   900 -> 901: killed
mutation_010_boundary_L10.ek9  BOUNDARY  line 10  250 -> 249: killed
mutation_011_boundary_L10.ek9  BOUNDARY  line 10  250 -> 251: killed
mutation_012_cmp_flip_L20.ek9  CMP_FLIP  line 20  <= -> <: survived
mutation_013_cmp_flip_L22.ek9  CMP_FLIP  line 22  > -> <: killed
mutation_014_cmp_flip_L22.ek9  CMP_FLIP  line 22  > -> >=: survived
mutation_015_arith_swap_L26.ek9 ARITH_SWAP line 26  + -> -: killed

Every change to a price or to the surcharge was caught, because a test asserts each price. The survivors are the boundaries: the limit between a letter and a parcel (500) and the limit above which a parcel is heavy (5000), neither of which any test uses, and the two comparison changes that move a weight across one of those limits, <= to < on line 20 and > to >= on line 22. That project has full coverage and a modest HEX score (Quality Built In), and six of its sixteen mutants survive. The mutants that survive are the tests you still owe, and unlike a variety figure they are specific: each names a line and a change.

Fuzzing the compiler

-fuzz is for people who work on the compiler, and for anyone who wants to know how it is tested. It generates programs from the grammar and mutates the corpus, feeds them to the compiler, and reports any that make the compiler itself crash. A one minute run:

EK9 Grammar Fuzzer started. Duration: 1m Workers: 9
Crash files will be saved to: ./fuzz-crashes
Templates loaded: 1254 (S2: 20%, S3: 20%, S5: 20%)
Targeted files: 1122 (1818 mutations, S4: 20%)

PASS: 0 crashes, 0 known, 398/442 errors, 31,964 programs in 1m 0s
EK9     : EK9 running grammar-based fuzzer

A crash is saved under fuzz-crashes/ as a file you can compile again, and a run that finds one exits with 15, so a build can gate on it. -fuzz2 writes JSON and -fuzz6 an HTML dashboard. It tests the compiler, not your program; for your program the tools above are the ones.

Asking the compiler

Three commands answer most questions without leaving the terminal. ek9 -h <keyword> is the reference for a construct, a built-in type or an error code, ek9 -H lists every keyword, and ek9 -q <words> searches a corpus of questions and answers that carry compiled examples; -qm adds a comparison with another language. When you want the compiler to explain a diagnostic at length, add -E3 to the compile. The exit codes a script can rely on are in ek9 -h: 8 for a compile failure, 11 for a failed test, 12 for coverage below the bar and 15 for a fuzz crash.

For the network types, the next chapter to reread is Networking: TCP and UDP; for packaging what you have built, Packaging and Sharing Your Program; for the places where the compiler stops you and why, Quality Built In and The Payoffs: Liskov and Hoare.