Why does EK9 have a built-in test framework instead of using JUnit or pytest?

← Testing · Ref: Q804

EK9 builds testing into the compiler and language, eliminating the need for JUnit, pytest, or any external test framework. This is a deliberate design choice.

WHY BUILT-IN

External frameworks create a gap between what the compiler knows and what the test runner knows. EK9 closes that gap:
- The compiler validates test quality at compile time (no empty tests, no orphan assertions)
- Coverage instrumentation is built into the compilation pipeline
- No dependency management for test libraries
- No version conflicts between test framework and language

COMPARISON

Java: JUnit 5 is a separate dependency, needs Maven/Gradle integration, assertion methods imported from org.junit.jupiter.api. Python: pytest is pip-installed, uses assert rewriting magic. Rust: #[test] is built-in but coverage requires external tools (tarpaulin). Go: testing package is built-in — closest to EK9's approach.

EK9 goes further than Go: compile-time test quality validation (E81007 empty test, E81011 orphan assertion), built-in coverage with 80% threshold, HTML dashboard, and test grouping for parallel/sequential control.

THE QUALITY ADVANTAGE

Because the compiler sees your tests, it can enforce quality:
- Tests must actually verify something (assert, assertThrows, or expected_output.txt)
- Assertions must be reachable from a @Test program (call graph analysis)
- Assertions cannot appear in production code
- All code quality metrics (complexity, cohesion, coupling) apply to test code too

No external framework can provide this level of integration.

See Q155 for writing tests. See Q206 for coverage. See Q207 for output formats.

Example

defines module qa.testdeep.why.builtin

  defines function

    calculate() as pure
      -> n as Integer
      <- rtn as Integer: n * n

  defines program

    // === EK9: NO FRAMEWORK NEEDED ===
    // No imports, no dependencies, no build plugin.
    // Just @Test and assert.

    @Test
    SquareOfFiveTest()
      result <- calculate(5)
      assert result == 25

    // === GROUPED TESTS RUN SEQUENTIALLY ===

    @Test: "math"
    SquareOfThreeTest()
      result <- calculate(3)
      assert result == 9

    @Test: "math"
    SquareOfZeroTest()
      result <- calculate(0)
      assert result == 0

Common mistakes

E81007 — A @Test program must validate something: assert statements, assertThrows, or an expected_output.txt file. Printing output without a companion expected file is an empty test. See ek9 -h E81007 for details.

Incorrect:

@Test
    EmptyTest()
      result <- calculate(5)
      stdout <- Stdout()
      stdout.println($result)

Correct:

    @Test
    SquareOfFiveTest()
      result <- calculate(5)
      assert result == 25

Coming from another language?

Java: JUnit 5 requires org.junit.jupiter dependency, @Test annotation, Assertions.* static imports, build tool plugin. Python: pytest requires pip install, conftest.py configuration, -v flags. Rust: #[test] is built-in but cargo-tarpaulin needed for coverage. Go: testing.T built-in, go test -cover for coverage. EK9: everything built in — @Test, assert, assertThrows, coverage, HTML reports, quality validation.

Keywords: comparison, migrate, quality, testing, framework, built-in, coverage, pytest, junit, advantage