How do coverage thresholds and exit codes work in EK9?

← Testing · Ref: Q811

EK9 enforces an 80% coverage threshold as a quality gate. Test exit codes distinguish between test failures and coverage failures.

EXIT CODES

The test runner returns specific exit codes:

  0   All tests passed AND coverage >= 80%
  11  One or more tests failed
  12  All tests passed BUT coverage < 80%

This means a CI/CD pipeline can distinguish between:
- Everything good (0)
- Broken code (11) — fix the failing tests first
- Insufficient testing (12) — add more tests

THE 80% THRESHOLD

EK9 enforces 80% line coverage by default. This is not configurable per-project — it is a language-level quality gate. If your tests pass but only cover 60% of the code, the build fails with exit code 12.

WHY 80%

Industry research shows 80% is the sweet spot:
- Below 80%: significant untested paths likely contain bugs
- Above 80%: diminishing returns, risk of gaming metrics
- 100%: usually not worth the effort for the last 20%

COVERAGE MODES

Three instrumentation modes:

  SET     records whether each line was executed (default, lowest overhead)
  COUNT   counts how many times each line ran (identifies hot paths)
  ATOMIC  thread-safe counting for concurrent tests

VIEWING COVERAGE

  ek9 -t4 myApp.ek9    JSON output with coverage data
  ek9 -t5 myApp.ek9    verbose coverage breakdown per function
  ek9 -t6 myApp.ek9    interactive HTML dashboard with source highlighting

The HTML dashboard (-t6) shows coverage per function, complexity badges, and readability scores.

CI/CD INTEGRATION

  ek9 -t2 myApp.ek9    JSON for programmatic parsing
  ek9 -t3 myApp.ek9    JUnit XML for CI dashboards (Jenkins, GitLab)
  echo $?              check exit code: 0=pass, 11=fail, 12=low coverage

See Q206 for coverage basics. See Q207 for output formats. See Q804 for why built-in testing.

Example

defines module qa.testdeep.coverage.thresholds

  defines function

    classify() as pure
      -> n as Integer
      <- rtn as String?

      if n > 0
        rtn: "positive"
      else if n < 0
        rtn: "negative"
      else
        rtn: "zero"

  defines program

    // === COVERAGE THRESHOLD DEMO ===
    // Tests must cover >= 80% of code to pass.
    // Exit code 0  = pass + coverage OK
    // Exit code 11 = test failure
    // Exit code 12 = tests pass but coverage < 80%

    @Test
    ClassifyPositiveTest()
      result <- classify(5)
      assert result == "positive"

    @Test
    ClassifyNegativeTest()
      result <- classify(-3)
      assert result == "negative"

    @Test
    ClassifyZeroTest()
      result <- classify(0)
      assert result == "zero"

    // All three branches covered = good coverage.
    // Missing any branch risks exit code 12.
Other ways to ask this
  • What happens if my EK9 test coverage is below 80%?
  • What do EK9 test exit codes mean?
  • How does EK9 enforce minimum code coverage?

Coming from another language?

Java: JaCoCo with Maven enforcer plugin, configurable thresholds per package. Python: coverage.py with --fail-under=80. Rust: cargo-tarpaulin with --fail-under 80. Go: go test -cover shows percentage but no threshold enforcement built in. EK9: 80% threshold enforced by the compiler, exit code 12 for insufficient coverage, no configuration needed.

Keywords: percent, gate, threshold, enforce, exit, code, eighty, coverage, ci, pipeline