How do I measure test coverage in EK9?

← Testing · Ref: Q206

EK9 has built-in test coverage instrumentation. Use the -tC flag to enable coverage; a fixed 80% threshold is enforced.

COVERAGE FLAG

Run tests with coverage:

  ek9 -tC myTests.ek9

Or: ek9 -t4 myTests.ek9 (detailed coverage report listing uncovered items)

WHAT IS RECORDED

Coverage records whether each line was executed. There is no flag that selects any other counting mode.

THRESHOLD

The coverage threshold is 80% and is fixed - there is no setting to change it. When coverage falls below it the tests themselves still pass, but the run reports E83001 and exits with code 12 (all tests passed but less than 80% coverage).

OUTPUT FORMATS WITH COVERAGE

  -t4  Detailed report that lists the uncovered items
  -t5  Verbose coverage listing both covered and uncovered items
  -t6  HTML dashboard with visual coverage

See Q207 for all output formats. See Q155 for basic testing. See Q157 for running tests. See Q321 for the quality report dashboard.

Example

defines module qa.testdeep.coverage

  defines function

    addNumbers() as pure
      ->
        a as Integer
        b as Integer
      <-
        rtn as Integer: a + b

    isPositive() as pure
      -> n as Integer
      <- rtn as Boolean: n > 0

  defines program

    CoverageDemo()
      stdout <- Stdout()

      // === FUNCTIONS TO COVER ===

      result <- addNumbers(3, 4)
      stdout.println(`3 + 4 = ${result}`)

      positive <- isPositive(5)
      stdout.println(`5 is positive: ${positive}`)

      negative <- isPositive(-1)
      stdout.println(`-1 is positive: ${negative}`)

      // === COVERAGE ===
      // Run with: ek9 -tC myTests.ek9
      // Each line is recorded as executed or not.

      stdout.println("Coverage: each line is recorded as executed or not")
      stdout.println("Threshold: 80% (fixed, E83001 and exit code 12 when missed)")
      stdout.println("Formats: -t4 uncovered items, -t5 verbose, -t6 HTML")

Common mistakes

E08091 — The parameter n is declared but never used in the function body. All parameters must be referenced in the implementation. See ek9 -h E08091 for details.

Incorrect:

isPositive() as pure
      -> n as Integer
      <- rtn as Boolean: true

Correct:

isPositive() as pure
      -> n as Integer
      <- rtn as Boolean: n > 0

Coming from another language?

Java: JaCoCo or Cobertura external tools. Python: coverage.py with pytest-cov. Rust: cargo-tarpaulin or llvm-cov. Go: go test -cover built-in. Kotlin: JaCoCo via Gradle. EK9: built-in coverage with a fixed 80% threshold (E83001, exit code 12 when missed) and an HTML dashboard.

Keywords: set, test, report, percent, instrument, coverage, threshold, html