Where do crash files and generated tests go in EK9?

← Fuzzing and Mutation Testing · Ref: Q757

EK9 fuzzing tools write output to predictable locations with options to control overwriting and reproducibility.

CRASH FILES (-fuzz)
Location: ./fuzz-crashes/
When the compiler crashes on a generated file, the full generated source is saved here (it is not minimised) together with a .log file holding the stack trace. Each name is made of a prefix, a timestamp and a counter, for example compiler-crash-20261004-101500-1.ek9 beside compiler-crash-20261004-101500-1.log.

GENERATED TESTS (-fuzztest)

  ek9 -fuzztest source.ek9 dev/generated.ek9

The second argument is the output file path, and it must be a dev source (a dev/ directory or a QA####_dev_ file) or it is refused with E81015. By default, the command errors if the output file already exists. Use -overwrite to replace it.

MUTATION VARIANTS (-fuzzmutate)

  ek9 -fuzzmutate source.ek9 mutations/

The second argument is the output directory. Contains:
- manifest.txt lists each variant with its mutation category and location.
- mutation_<nnn>_<category>_L<line>.ek9, for example mutation_001_bool_logic_L70.ek9, are the individual mutated files.
By default, errors if the directory already exists. Use -overwrite to replace.

CONTROLLING OUTPUT

  -overwrite     Allow replacing existing output file or directory.
  -seed <n>      Reproducible generation (same seed produces same output).
  -n <n>         Maximum number of candidates (-fuzztest default 200, -fuzzmutate default 100).

CI WORKFLOW

In CI, use -overwrite so that each build produces fresh output:

  ek9 -fuzztest source.ek9 dev/generated-tests.ek9 -overwrite
  ek9 -fuzzmutate source.ek9 mutations/ -overwrite -test tests.ek9

See Q749 for fuzzing overview. See Q750 for running the fuzzer. See Q753 for mutation testing. See Q754 for test generation. See Q2 for compile and run.

Example

defines module qa.fuzzingandmutation.crashfiles

  <?-
    Demonstrates code patterns that might produce crash
    files when fuzzed, and test output when generated.
    The output management options ensure CI repeatability.
  -?>

  defines class

    Stack
      items as List of String: List() of String

      push()
        -> item as String
        this.items += item

      pop()
        <- popped as String: String()

        count <- length this.items
        zero <- 0
        emptyDefault <- String()
        if count > zero
          lastIndex <- count - 1
          topItem <- this.items.getOrDefault(lastIndex, emptyDefault)
          if topItem?
            popped: topItem
            this.items -= topItem

      peek() as pure
        <- topItem as String: String()

        count <- length this.items
        zero <- 0
        emptyDefault <- String()
        if count > zero
          lastIndex <- count - 1
          topItem :=? this.items.getOrDefault(lastIndex, emptyDefault)

      size() as pure
        <- count as Integer: length this.items

      operator $ as pure
        <- rtn as String: $this.items

      default operator ?

  defines program

    CrashFilesDemo()
      stdout <- Stdout()

      stack <- Stack()
      stack.push("first")
      stack.push("second")
      stack.push("third")
      stdout.println(`Stack: ${stack}`)

      popped <- stack.pop()
      stdout.println(`Popped: ${popped}`)
      stdout.println(`Peek: ${stack.peek()}`)
      stdout.println(`Size: ${stack.size()}`)
Other ways to ask this
  • What is the ./fuzz-crashes/ directory?
  • How do I manage EK9 fuzzer output files?
  • What options control EK9 fuzz output locations?

Coming from another language?

Java: Jazzer saves crashes to a corpus directory, no structured manifest. Python: Atheris writes to crash-* files in current directory. Rust: cargo-fuzz saves to fuzz/artifacts/<target>/. Go: go test -fuzz saves to testdata/fuzz/<FuzzTestName>/. EK9: crash files in ./fuzz-crashes/, generated tests to specified file, mutations to specified directory with manifest.txt, all controllable via -overwrite, -seed, -n.

Keywords: output, reproducible, manifest, crash, seed, location, directory, files, fuzz-crashes, mutation, overwrite, variant