How does EK9 enforce test quality at compile time?

← Testing · Ref: Q809

EK9 validates test quality during compilation. Tests that verify nothing or have unreachable assertions are compiler errors, not warnings.

E81007: EMPTY TEST

A @Test program must validate something. If it has no assert statements, no assertThrows, and no expected_output.txt companion file, the compiler rejects it:

  @Test
  EmptyTest()
    result <- calculate(5)
    stdout <- Stdout()
    stdout.println($result)   // prints but verifies nothing

This fails with E81007. Fix by adding assert result == 25 or providing expected_output.txt.

E81011: ORPHAN ASSERTION

Assertions must be reachable from at least one @Test program through the call graph:

  helperFunction()
    assert someCondition()    // orphan — no @Test calls this

If no @Test program calls helperFunction() directly or indirectly, the assert is unreachable and the compiler reports E81011. This catches forgotten test wiring.

E81012: PRODUCTION ASSERTION

Assert and assertThrows are only valid in @Test programs or functions called from @Test programs. Using them in production code is a compiler error.

QUALITY METRICS APPLY TO TESTS

Test code is held to the same quality standards as production code:
- Complexity limits (cyclomatic, cognitive, nesting)
- Cohesion and coupling checks
- Variable naming rules (no temp, flag, data, value)
- Reference ordering (alphabetical)
- Unused parameter detection

WHY THIS MATTERS

In Java/Python, a test that calls code but asserts nothing passes silently — false coverage that hides bugs. EK9 makes this a compile error. You cannot have a test that does not test.

See Q804 for why built-in testing. See Q155 for writing tests. See Q310 for code quality metrics.

Example

defines module qa.testdeep.quality.enforcement

  defines function

    compute() as pure
      -> n as Integer
      <- rtn as Integer: n * n

  defines program

    // === VALID TESTS: ASSERT SOMETHING ===

    @Test
    ComputeSquareTest()
      result <- compute(7)
      assert result == 49

    @Test
    ComputeZeroTest()
      result <- compute(0)
      assert result == 0

    TestQualityDemo()
      stdout <- Stdout()
      stdout.println("E81007: empty test (no assertions)")
      stdout.println("E81011: orphan assertion (unreachable)")
      stdout.println("E81012: assert in production code")
      stdout.println("All quality metrics apply to test code")

Common mistakes

E81007 — A @Test program with no assert, no assertThrows, and no expected_output.txt is an empty test — it executes code but validates nothing. Add assertions or provide an expected output file. See ek9 -h E81007 for details.

Incorrect:

@Test
    NoVerification()
      result <- compute(10)
      stdout <- Stdout()
      stdout.println($result)

Correct:

    @Test
    ComputeSquareTest()
      result <- compute(7)
      assert result == 49

E81011 — An assert statement must be reachable from at least one @Test program through the call graph. An unreachable assertion is dead test code. See ek9 -h E81011 for details.

Incorrect:

verifyResult()
      -> value as Integer
      assert value > 0
    // No @Test program calls verifyResult

Correct:

    @Test
    ComputeSquareTest()
      result <- compute(7)
      assert result == 49

    @Test
    ComputeZeroTest()

Coming from another language?

Java: JUnit has no compile-time quality checks — empty tests pass silently. SonarQube can detect some patterns but is optional. Python: pytest allows empty tests, flake8 cannot detect missing assertions. Rust: empty #[test] functions compile and pass. Go: empty Test functions compile and pass. EK9: compile-time rejection of empty tests (E81007), orphan assertions (E81011), and production assertions (E81012).

Keywords: empty, assertion, enforce, E81007, quality, compile, validate, orphan, E81012, E81011