Code Coverage in EK9

EK9 collects code coverage every time you run your tests, and it always enforces it: below the 80% threshold the test runner fails with exit code 12, even when every test passes.

The -t6 quality report: the Quality at a Glance band, complexity budget tracks, and the coverage detail beneath

Threshold Enforcement

The compiler inserts coverage probes at every control flow point:

Every -t run collects them and checks the threshold, whether or not you asked for a coverage report. Here one test reaches one of three temperature functions: 

#!ek9
defines module site.coverage.shortfall

  defines function

    celsiusToFahrenheit() as pure
      -> celsius as Float
      <- fahrenheit as Float: celsius * 9.0 / 5.0 + 32.0

    fahrenheitToCelsius() as pure
      -> fahrenheit as Float
      <- celsius as Float: (fahrenheit - 32.0) * 5.0 / 9.0

    describe() as pure
      -> celsius as Float
      <- rtn as String: celsius < 0.0 <- "freezing" else "not freezing"

Download coverage_shortfall.ek9

#!ek9
defines module site.coverage.shortfall.tests

  references
    site.coverage.shortfall::celsiusToFahrenheit

  defines program

    @Test
    BoilingPoint()
      assert celsiusToFahrenheit(100.0) == 212.0

Download coverage_shortfall_tests.ek9

The test passes and the run still fails with E83001 and exit code 12, naming what no test reached: 

$ ek9 -t coverage_shortfall.ek9
ℹ️ Found 1 test:
  1 assert (unit tests with assertions)

Executing 1 test...

✅ PASS site.coverage.shortfall.tests::BoilingPoint [Assert] (nms)

Summary: 1 passed, 0 failed (1 total)
Types: 1 assert
Duration: nms

❌ E83001: Code coverage 27.3% is below required threshold 80%
   Methods:  33.3% (1/3)
   Lines:    33.3% (3/9)
   Branches: 0.0% (0/2)

   Uncovered items:
     2 method(s)
       • site.coverage.shortfall::fahrenheitToCelsius (./coverage_shortfall.ek9:10)
       • site.coverage.shortfall::describe (./coverage_shortfall.ek9:14)
     2 branch(es)

   Use -t4 or -t5 for detailed coverage report.
ek9: coverage below the required threshold (exit 12)

Why EK9 does it this way. Passing tests are not enough; the code must be tested. Elsewhere coverage is an optional report that someone has to remember to read. In EK9 a CI/CD pipeline catches too little coverage even when nobody asked for a report.

Coverage Output Formats

Add -tC to show the coverage results, in the same format as the test output: 

ek9 -t -tC main.ek9         # Human-readable coverage summary
ek9 -t0 -tC main.ek9        # Terse: one more line, Methods: N% Lines: N% Branches: N%
ek9 -t2 -tC main.ek9        # JSON output to files
ek9 -t3 -tC main.ek9        # JUnit XML + JaCoCo XML to files

The examples below run the same functions with tests that reach every function and both sides of the branch: 

#!ek9
defines module site.coverage.report

  defines function

    celsiusToFahrenheit() as pure
      -> celsius as Float
      <- fahrenheit as Float: celsius * 9.0 / 5.0 + 32.0

    fahrenheitToCelsius() as pure
      -> fahrenheit as Float
      <- celsius as Float: (fahrenheit - 32.0) * 5.0 / 9.0

    describe() as pure
      -> celsius as Float
      <- rtn as String: celsius < 0.0 <- "freezing" else "not freezing"

Download coverage_report.ek9

#!ek9
defines module site.coverage.report.tests

  references
    site.coverage.report::celsiusToFahrenheit
    site.coverage.report::describe
    site.coverage.report::fahrenheitToCelsius

  defines program

    @Test
    BoilingPoint()
      assert celsiusToFahrenheit(100.0) == 212.0

    @Test
    RoundTrip()
      assert fahrenheitToCelsius(celsiusToFahrenheit(37.0)) == 37.0

    @Test
    BothSidesOfFreezing()
      assert describe(-5.0) == "freezing"
      assert describe(20.0) == "not freezing"

Download coverage_report_tests.ek9

$ ek9 -t -tC coverage_report.ek9
ℹ️ Found 3 tests:
  3 assert (unit tests with assertions)

Executing 3 tests...

✅ PASS site.coverage.report.tests::BoilingPoint [Assert] (nms)
✅ PASS site.coverage.report.tests::BothSidesOfFreezing [Assert] (nms)
✅ PASS site.coverage.report.tests::RoundTrip [Assert] (nms)

Summary: 3 passed, 0 failed (3 total)
Types: 3 assert
Duration: nms

📊 Coverage Summary
  ✅ Methods:  100.0% (3/3)
  ✅ Lines:    100.0% (9/9)
  ✅ Branches: 100.0% (2/2)

  ✅ Overall: 100.00% (11/11 probes hit)

📦 By Module:
  ✅ site.coverage.report: 100.00% (11/11)
$ ek9 -t0 -tC coverage_report.ek9
3 tests: 3 passed, 0 failed (3 assert)
Methods: 100% Lines: 100% Branches: 100%

File Output for Machine Formats

With a machine format (-t2 or -t3) and -tC, the results are written to files in the .ek9/ directory:

Flags Test Results Coverage Results
-t2 -tC .ek9/test-results.json .ek9/coverage.json
-t3 -tC .ek9/test-results.xml (JUnit) .ek9/coverage.xml (JaCoCo)

A short summary naming the files is printed to stdout: 

$ ek9 -t2 -tC coverage_report.ek9

✅ 3 passed, 0 failed (3 tests) [nms]
Coverage: 100% methods, 100% lines, 100% branches

Reports written to:
  project-directory/./.ek9/test-results.json
  project-directory/./.ek9/coverage.json

.ek9/coverage.xml is JaCoCo XML, so tools that read that format, such as SonarQube, Codecov and Coveralls, can take it.

Coverage JSON Format

$ ek9 -t2 -tC coverage_report.ek9 && cat .ek9/coverage.json
{
  "coverage" : {
    "methods" : {
      "percentage" : 100.0,
      "covered" : 3,
      "total" : 3
    },
    "lines" : {
      "percentage" : 100.0,
      "covered" : 9,
      "total" : 9
    },
    "branches" : {
      "percentage" : 100.0,
      "covered" : 2,
      "total" : 2
    },
    "overall" : 100.0,
    "probesHit" : 11,
    "probesTotal" : 11,
    "probeTypes" : {
      "BRANCH_FALSE" : {
        "hit" : 1,
        "total" : 1
      },
      "BRANCH_TRUE" : {
        "hit" : 1,
        "total" : 1
      },
      "FUNCTION_ENTRY" : {
        "hit" : 3,
        "total" : 3
      },
      "STATEMENT" : {
        "hit" : 6,
        "total" : 6
      }
    },
    "modules" : {
      "site.coverage.report" : {
        "coverage" : 100.0,
        "hit" : 11,
        "total" : 11
      }
    },
    "metrics" : {
      "avgComplexity" : 1.67,
      "maxComplexity" : 3,
      "totalComplexity" : 5,
      "coveredComplexity" : 5,
      "avgCognitive" : 0.0,
      "maxCognitive" : 0
    },
    "readability" : {
      "./coverage_report.ek9" : "5"
    }
  }
}

JaCoCo XML Format

$ ek9 -t3 -tC coverage_report.ek9 && cat .ek9/coverage.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE report PUBLIC "-//JACOCO//DTD Report 1.1//EN" "report.dtd">
<report name="EK9 Coverage">
  <counter type="METHOD" missed="0" covered="3"/>
  <counter type="LINE" missed="0" covered="9"/>
  <counter type="BRANCH" missed="0" covered="2"/>
  <counter type="COMPLEXITY" missed="0" covered="5"/>
  <package name="site/coverage/report">
    <counter type="BRANCH" missed="0" covered="11"/>
  </package>
</report>

Verbose Coverage (-t4, -t5)

ek9 -t4 main.ek9       # Coverage detail: lists the uncovered items, writes .ek9/coverage-detail.json
ek9 -t5 main.ek9       # Verbose: lists covered AND uncovered items, writes .ek9/coverage-verbose.json

-t5 lists, for each kind of probe, what the tests reached and what they did not, each with its file and line. On the under-tested temperature functions from above: 

$ ek9 -t5 coverage_shortfall.ek9
ℹ️ Found 1 test:
  1 assert (unit tests with assertions)

Executing 1 test...

✅ PASS site.coverage.shortfall.tests::BoilingPoint [Assert] (nms)

Summary: 1 passed, 0 failed (1 total)
Types: 1 assert
Duration: nms

📊 Coverage Summary
  ❌ Methods:  33.3% (1/3)
  ❌ Lines:    33.3% (3/9)
  ❌ Branches: 0.0% (0/2)

  ❌ Overall: 27.27% (3/11 probes hit)

📦 By Module:
  ❌ site.coverage.shortfall: 27.27% (3/11)

📄 Verbose coverage report written to: .ek9/coverage-verbose.json

✅ Covered Items:
  ✅ Methods (1):
    • site.coverage.shortfall::celsiusToFahrenheit (./coverage_shortfall.ek9:6)
  ✅ Statements (2):
    • ./coverage_shortfall.ek9 (lines: 7, 8)

❌ Uncovered Items:
  ⚠️ Methods (2):
    • site.coverage.shortfall::fahrenheitToCelsius (./coverage_shortfall.ek9:10)
    • site.coverage.shortfall::describe (./coverage_shortfall.ek9:14)
  ⚠️ Branches (2):
    • ./coverage_shortfall.ek9:16 (BRANCH_FALSE, BRANCH_TRUE)
  ⚠️ Statements (4):
    • ./coverage_shortfall.ek9 (lines: 11, 12, 15, 16)

Input-variety (numerator, measurement-only):
  site.coverage.shortfall::celsiusToFahrenheit       driven 1 cell(s), asserted 1: p0: POSITIVE

❌ E83001: Code coverage 27.3% is below required threshold 80%
   Methods:  33.3% (1/3)
   Lines:    33.3% (3/9)
   Branches: 0.0% (0/2)

   Uncovered items:
     2 method(s)
       • site.coverage.shortfall::fahrenheitToCelsius (./coverage_shortfall.ek9:10)
       • site.coverage.shortfall::describe (./coverage_shortfall.ek9:14)
     2 branch(es)

   Use -t4 or -t5 for detailed coverage report.
ek9: coverage below the required threshold (exit 12)

It writes the same information to .ek9/coverage-verbose.json: 

$ ek9 -t5 coverage_shortfall.ek9 && cat .ek9/coverage-verbose.json
{
  "coverage" : {
    "overall" : 27.272727272727273,
    "methods" : 33.333333333333336,
    "lines" : 33.333333333333336,
    "branches" : 0.0
  },
  "coveredMethods" : [ {
    "name" : "celsiusToFahrenheit",
    "module" : "site.coverage.shortfall",
    "file" : "./coverage_shortfall.ek9"
  } ],
  "uncoveredMethods" : [ {
    "name" : "fahrenheitToCelsius",
    "module" : "site.coverage.shortfall",
    "file" : "./coverage_shortfall.ek9"
  }, {
    "name" : "describe",
    "module" : "site.coverage.shortfall",
    "file" : "./coverage_shortfall.ek9"
  } ],
  "coveredBranches" : [ ],
  "uncoveredBranches" : [ {
    "type" : "BRANCH_FALSE",
    "module" : "site.coverage.shortfall",
    "file" : "./coverage_shortfall.ek9",
    "line" : 16
  }, {
    "type" : "BRANCH_TRUE",
    "module" : "site.coverage.shortfall",
    "file" : "./coverage_shortfall.ek9",
    "line" : 16
  } ],
  "metrics" : {
    "avgComplexity" : 1.7,
    "maxComplexity" : 3,
    "avgCognitive" : 0.0,
    "maxCognitive" : 0
  },
  "highComplexityFunctions" : [ ],
  "readability" : {
    "./coverage_shortfall.ek9" : "5"
  }
}

-t4 writes .ek9/coverage-detail.json instead: only the uncovered items, with complexity and readability. Read it when you are acting on a coverage gap; its fields are in -t4: Coverage Detail JSON.

Code Quality Metrics

Coverage counts lines and branches. The report also measures every function for the complexity that makes code hard to maintain.

For AI Assistants: See Metrics Reference for the threshold tables and recommended actions, and the workflow templates for improving quality step by step.

Available Metrics

Metric Range Description
Cyclomatic Complexity (CC) 1-50+ The number of linearly independent paths through the code. Each if, switch case, for, while and exception handler adds 1. Aim for under 10 per function.
Cognitive Complexity 0-50+ The effort needed to understand the code. It penalises nesting, breaks in linear flow and boolean complexity, so it follows how people read code more closely than CC does. Aim for under 15.
Nesting Depth 0-10+ The deepest stack of nested control structures. Deep nesting is code to refactor. Aim for under 4.
Statement Count 0-100+ The number of executable statements. Large functions are harder to test and to maintain. Aim for under 20 per function.
ARI Readability 1-20+ The Automated Readability Index: lexical complexity, from identifier length and statement density. Lower is simpler to read. Measured per file.

Compile-time enforcement. The targets above are advice. The compiler enforces hard limits that sit above them, in production and test code alike: cyclomatic complexity over 45 (E11010), cognitive complexity over 35 (E11021), nesting deeper than 6 (E11011) and more than 150 statements in a function (E11012) do not compile. Code Quality has every threshold, its error code and the reasoning behind it.

Where Metrics Appear

Complexity in the Quality Line

In a source view, each function has a quality line above its declaration, written like a comment. It starts with the complexity: 

// CC:6
isNumeric()
  -> text as String
  <- result as Boolean: false
  // ... the body

The colour follows the cyclomatic complexity:

Click the CC token for the full figures: cyclomatic, cognitive and nesting depth.

Readability (ARI) Scores

The Automated Readability Index measures lexical complexity: how easy the words in your code are to read, not how easy the logic is to follow. Lower scores mean simpler identifier names and statements.

Each file's score is in the Readability column of its module page, and on the file's summary page with its band in words:

Note: ARI is reported, never enforced; no error fires on it. Domain terms often need longer identifiers. ARI Readability covers it in full, with the domain considerations.

Input Variety

Coverage answers one question: did this line run? It cannot tell you whether the line ran with input that mattered, or whether anything checked the result. A project can sit at 100% coverage with no test asserting anything, and coverage reports that as a perfect score.

Input variety answers the other two questions. For every construct the compiler works out a floor: the value-classes its parameters can meaningfully take. The test run then records which of them were reached, and which had their result checked.

The floor is 1-wise: each value-class must be driven at least once, not every combination of them. A function taking a three-member enumeration and a Boolean has a floor of seven (4 + 3), not twelve, so full variety is reachable rather than combinatorially hopeless.

Value-classes and why the parameter type matters

How well a construct can be measured depends on what its parameter types say. An enumeration declares its own value-classes: its members, plus unset. Boolean has exactly three: true, false and unset. These are known exactly.

A String's meaningful values have to be guessed, because the type does not say what they are. Constructs taking one are reported apart, as approximate, and that list is a work list. Replace a bare String parameter with an enumeration or a constrained type and the construct moves into the measured group, with the type carrying the meaning, as EK9 asks everywhere else.

The verification target: a complete green outer ring for 100% coverage, a half-length red ring inside it for 49.8% driven, and an empty innermost ring for 0% asserted

The three rings on a project at 100% coverage that asserts nothing: the outer ring complete, the middle ring half-filled and the centre empty. Hover any ring for the counts behind it.

Look for a wide gap between driven and asserted. It means code runs as scaffolding for other tests, with nothing verifying what it did.

Change Risk (HEX)

HEX fuses the metrics above into one question: how costly is a change here likely to be? It is the product of two factors:

Neither is enough alone. Complex code that is thoroughly verified is not especially risky to change, and neither is trivial code that nothing checks. Risk is where the two meet.

Why EK9 does it this way. HEX never gates a build. Coverage is the enforced threshold; HEX is an indicator, for a person or an AI deciding where to be careful. It is there to counter the idea that a coverage percentage alone says whether code is safe to change.

Reading the score

The score runs from 0 to 100, with bands at 25 and 50 (low, moderate, high). Read the typical score and the worst single construct together:

A low score means low change-risk, not "well tested". A trivial construct scores low because there is little to go wrong, not because it is guarded. So the report shows both factors on every construct: hazard 0% × exposure 100% and hazard 20% × exposure 0% both fuse to zero, and they are opposite situations. The first is untested trivial code; the second is guarded code.

How much of your project could be measured

A score over a known input space and a score over a guessed one are different measurements, and their average means neither. So constructs are reported in three groups:

HTML Coverage Reports (-t6)

ek9 -t6 main.ek9       # Generate HTML coverage report
ek9 -t6p main.ek9      # HTML report with profiling (flame graph, hot function table)
$ ek9 -t6 coverage_report.ek9
ℹ️ Found 3 tests:
  3 assert (unit tests with assertions)

Executing 3 tests...

✅ PASS site.coverage.report.tests::BoilingPoint [Assert] (nms)
✅ PASS site.coverage.report.tests::BothSidesOfFreezing [Assert] (nms)
✅ PASS site.coverage.report.tests::RoundTrip [Assert] (nms)

Summary: 3 passed, 0 failed (3 total)
Types: 3 assert
Duration: nms

📊 Coverage Summary
  ✅ Methods:  100.0% (3/3)
  ✅ Lines:    100.0% (9/9)
  ✅ Branches: 100.0% (2/2)

  ✅ Overall: 100.00% (11/11 probes hit)

📦 By Module:
  ✅ site.coverage.report: 100.00% (11/11)

📊 HTML coverage report generated: .ek9/coverage/index.html
   1 source file(s) with line-by-line highlighting

This writes a coverage website to .ek9/coverage/, its views linked to each other.

Report Structure

.ek9/coverage/
├── index.html                          # Dashboard with project overview
├── input-variety.json                  # The input-variety figures the pages show
├── modules/                            # Module detail pages
│   └── site_coverage_report.html
├── files/                              # Source file views
│   ├── __coverage_report_ek9.html          # Full source with highlighting
│   └── summary/                        # File summary views
│       └── __coverage_report_ek9.html
├── coverage.css                        # Styles with dark mode support
├── coverage.js                         # Interactive features
├── EK9.png, EK9.svg                    # Branding
└── ek9favicon.png

Dashboard (index.html)

Every page (project, module and file) opens with the same band, so it reads the same wherever you have drilled to: what the tests verify, and what that means for changing this code.

Quality at a Glance

Quality at a Glance: a verification target of three concentric rings beside the HEX risk profile sunburst

The rings and the bars point in opposite directions on purpose. A ring is a target to fill: full is good. A budget bar is a ceiling to stay under: empty is good.

The rest of the dashboard

Complexity budget tracks

Complexity and cognitive complexity as budget bars, each showing the maximum as fill, the average as a marker, and the remaining headroom

Each metric has one track: the maximum as the bar, the average as a marker on it, because how the two relate is the finding.

The remaining headroom is stated in words beside each track. These limits are enforced: pass them and the code does not compile (E11010 complexity, E11021 cognitive complexity). Readability appears beside them as a value with no bar, because no error fires on it: it is measured and stated, never enforced.

Module and File Pages

Click any module to see the same band scoped to it, then the detail:

A file page carries the same rows. It states its HIGH count instead of repeating the charts: over a handful of constructs a chart says what the rows already say, and each row already carries its construct's score and both factors.

Source Code Views

Source code view with coverage highlighting, complexity badges, and profiling badges

The most detailed view is your source, highlighted, with the coverage data laid over it:

Interactive Features

Navigation Flow

Dashboard (index.html)
    │
    ├─╺ Attention Panel ╺ Module Page
    │
    └─╺ Module List ╺ Module Page (modules/myapp_utils.html)
                          │
                          ├─╺ Source File List ╺ File Summary
                          │                         │
                          │                         └─╺ Source View (files/__utils_ek9.html)
                          │
                          └─╺ Function List ╺ Source View (with line anchor)

Breadcrumbs lead back up: Source → Module → Dashboard.

See Also