What is fuzzing and how does EK9 support it?

← Fuzzing and Mutation Testing · Ref: Q749

EK9 has three built-in fuzzing modes integrated directly into the compiler. No external tools, no separate configuration.

THREE MODES

1. Compiler fuzzing (-fuzz): generates random EK9 source files and feeds them through the compiler to find parser and type-checker crashes. This tests the compiler itself.
2. Mutation testing (-fuzzmutate): takes your source file, applies semantic mutations (swap operators, flip comparisons, remove guards), and checks whether your tests detect the change. Surviving mutants reveal weak tests.
3. Test generation (-fuzztest): analyses your source, harvests types and functions, generates edge-case test programs, compiles them to verify correctness, and outputs the survivors as @Test programs.

SOURCE-LEVEL AND TYPE-AWARE

Unlike AFL or libFuzzer which operate on raw bytes, EK9 fuzzing understands the grammar and type system. Generated programs are syntactically valid EK9. Mutations respect operator semantics. Test generation uses type information to choose meaningful edge-case values.

WHY BUILT-IN

Fuzzing as an external tool means most teams never use it. By integrating it into the compiler, EK9 makes fuzzing a normal part of the development workflow, not a specialist activity.

See Q750 for running the fuzzer. See Q753 for mutation testing. See Q754 for test generation. See Q758 for comparison with AFL and libFuzzer. See Q155 for writing tests. See Q157 for running tests.

Example

defines module qa.fuzzingandmutation.overview

  <?-
    Code that benefits from all three fuzzing modes.
    The calculator operators would be flipped by mutation testing.
    The boundary checks would be probed by test generation.
    The parsing would be stressed by compiler fuzzing.
  -?>

  defines function

    safeDiv() as pure
      ->
        numerator as Float
        denominator as Float
      <- result as Float: 0.0

      zero <- 0.0
      if denominator <> zero
        result: numerator / denominator

    clamp() as pure
      ->
        low as Float
        high as Float
        input as Float
      <- result as Float: input

      if input < low
        result: low
      else if input > high
        result: high

  defines program

    FuzzingOverviewDemo()
      stdout <- Stdout()

      ten <- 10.0
      three <- 3.0
      stdout.println(`10 / 3 = ${safeDiv(ten, three)}`)

      zero <- 0.0
      stdout.println(`10 / 0 = ${safeDiv(ten, zero)}`)

      low <- 0.0
      high <- 100.0
      belowMin <- -5.0
      normal <- 50.0
      aboveMax <- 150.0
      stdout.println(`clamp(-5) = ${clamp(low, high, belowMin)}`)
      stdout.println(`clamp(50) = ${clamp(low, high, normal)}`)
      stdout.println(`clamp(150) = ${clamp(low, high, aboveMax)}`)

Coming from another language?

Java: no built-in fuzzer, use Jazzer (external, byte-level). Python: no built-in fuzzer, use Atheris or Hypothesis (external). Rust: cargo-fuzz wraps libFuzzer (external, byte-level). Go: go test -fuzz (built-in since 1.18, byte-level). JavaScript: no built-in, use jsfuzz (external). EK9: three built-in modes (compiler fuzz, mutation testing, test generation), all source-level and type-aware, integrated into the compiler.

Keywords: test, libFuzzer, fuzz, AFL, built-in, fuzzing, testing, generation, type-aware, source-level, mutation, compiler