How does fuzzing relate to code coverage and quality in EK9?

← Fuzzing and Mutation Testing · Ref: Q756

EK9 provides a complete quality loop: write tests, measure coverage, generate more tests, mutate to assess quality, and fuzz for robustness.

THE QUALITY LOOP (5 STEPS)

1. Write tests: create @Test programs for your functions.

  ek9 -t source.ek9

2. Check coverage: identify untested code.

  ek9 -tC source.ek9
  Coverage must reach 80% to package with -P.

3. Generate tests: fill coverage gaps automatically.

  ek9 -fuzztest source.ek9 generated.ek9
  Review generated tests, keep the useful ones.

4. Mutation testing: assess test quality.

  ek9 -fuzzmutate source.ek9 mutations/ -test tests.ek9
  Surviving mutants show where tests are too weak.

5. Compiler fuzzing: verify robustness.

  ek9 -fuzz 60
  Ensures the compiler handles edge cases in your patterns.

COVERAGE AND MUTATION TOGETHER

Coverage tells you what code is executed by tests. Mutation testing tells you whether tests actually verify behaviour. 100% coverage with weak assertions is meaningless. Mutation testing reveals this gap.

AI-ASSISTED WORKFLOW

Use -t2 (JSON test output) and -fuzz2 (JSON fuzz output) to feed results into AI analysis. The AI can identify patterns in surviving mutants and suggest targeted tests.

CONTINUOUS IMPROVEMENT

Run the quality loop regularly:

  Development: write tests, check coverage.
  Pre-commit: mutation testing on changed files.
  CI: short fuzz run as smoke test.
  Nightly: long fuzz run for deep coverage.

See Q749 for fuzzing overview. See Q753 for mutation testing. See Q754 for test generation. See Q155 for writing tests. See Q206 for test coverage. See Q310 for quality at compile time.

Example

defines module qa.fuzzingandmutation.qualityloop

  <?-
    A small library with tests that demonstrate the quality loop.
    Step 1: Write tests. Step 2: Check coverage.
    Step 3: Generate more. Step 4: Mutate. Step 5: Fuzz.
  -?>

  defines function

    maximum() as pure
      ->
        first as Integer
        second as Integer
      <- result as Integer: first

      if second > first
        result: second

    minimum() as pure
      ->
        first as Integer
        second as Integer
      <- result as Integer: first

      if second < first
        result: second

    absoluteValue() as pure
      -> number as Integer
      <- result as Integer: number

      zero <- 0
      if number < zero
        result: 0 - number

  defines program

    // === STEP 1: Write basic tests ===

    @Test
    MaximumTest()
      three <- 3
      seven <- 7
      assert maximum(three, seven) == 7
      assert maximum(seven, three) == 7

    @Test
    MinimumTest()
      three <- 3
      seven <- 7
      assert minimum(three, seven) == 3
      assert minimum(seven, three) == 3

    @Test
    AbsoluteValueTest()
      five <- 5
      negativeFive <- -5
      zero <- 0
      assert absoluteValue(five) == 5
      assert absoluteValue(negativeFive) == 5
      assert absoluteValue(zero) == 0

    // === Demo program showing the functions ===

    QualityLoopDemo()
      stdout <- Stdout()

      three <- 3
      seven <- 7
      stdout.println(`max(3, 7) = ${maximum(three, seven)}`)
      stdout.println(`min(3, 7) = ${minimum(three, seven)}`)
      stdout.println(`abs(-5) = ${absoluteValue(-5)}`)

Other ways to ask this

What is the EK9 quality loop for testing?
How do I combine fuzzing, mutation testing, and coverage in EK9?
How do I improve test quality in EK9 systematically?

Coming from another language?

Java: requires separate tools for each step (JUnit, JaCoCo, PIT, Jazzer) with different configs. Python: pytest + pytest-cov + mutmut + Atheris (all separate installs). Rust: cargo-test + tarpaulin + cargo-fuzz (no mutation tool). Go: go test + go test -cover + go test -fuzz (no mutation tool). EK9: complete quality loop in one tool: -t for tests, -tC for coverage, -fuzztest for generation, -fuzzmutate for mutation, -fuzz for robustness.

Keywords: test, fuzz, continuous, systematic, loop, generation, coverage, quality, mutation, improve, workflow, CI