Where do crash files and generated tests go in EK9?

← Fuzzing and Mutation Testing · Ref: Q757

EK9 fuzzing tools write output to predictable locations with options to control overwriting and reproducibility.

CRASH FILES (-fuzz)
Location: ./fuzz-crashes/
When the compiler crashes on a generated file, the full generated source is saved here (it is not minimised) together with a .log file holding the stack trace. Each name is made of a prefix, a timestamp and a counter, for example compiler-crash-20261004-101500-1.ek9 beside compiler-crash-20261004-101500-1.log.

GENERATED TESTS (-fuzztest)

  ek9 -fuzztest source.ek9 dev/generated.ek9

The second argument is the output file path, and it must be a dev source (a dev/ directory or a QA####_dev_ file) or it is refused with E81015. By default, the command errors if the output file already exists. Use -overwrite to replace it.

MUTATION VARIANTS (-fuzzmutate)

  ek9 -fuzzmutate source.ek9 mutations/

The second argument is the output directory. Contains:
- manifest.txt lists each variant with its mutation category and location.
- mutation_<nnn>_<category>_L<line>.ek9, for example mutation_001_bool_logic_L70.ek9, are the individual mutated files.
By default, errors if the directory already exists. Use -overwrite to replace.

CONTROLLING OUTPUT

  -overwrite     Allow replacing existing output file or directory.
  -seed <n>      Reproducible generation (same seed produces same output).
  -n <n>         Maximum number of candidates (-fuzztest default 200, -fuzzmutate default 100).

CI WORKFLOW

In CI, use -overwrite so that each build produces fresh output:

  ek9 -fuzztest source.ek9 dev/generated-tests.ek9 -overwrite
  ek9 -fuzzmutate source.ek9 mutations/ -overwrite -test tests.ek9

See Q749 for fuzzing overview. See Q750 for running the fuzzer. See Q753 for mutation testing. See Q754 for test generation. See Q2 for compile and run.

Example

#!ek9
defines module qa.fuzzingandmutation.crashfiles

  <?-
    Demonstrates code patterns that might produce crash
    files when fuzzed, and test output when generated.
    The output management options ensure CI repeatability.
  -?>

  defines class

    Stack
      items as List of String: List() of String

      push()
        -> item as String
        this.items += item

      pop()
        <- popped as String: String()

        count <- length this.items
        zero <- 0
        emptyDefault <- String()
        if count > zero
          lastIndex <- count - 1
          topItem <- this.items.getOrDefault(lastIndex, emptyDefault)
          if topItem?
            popped: topItem
            this.items -= topItem

      peek() as pure
        <- topItem as String: String()

        count <- length this.items
        zero <- 0
        emptyDefault <- String()
        if count > zero
          lastIndex <- count - 1
          topItem := this.items.getOrDefault(lastIndex, emptyDefault)

      size() as pure
        <- count as Integer: length this.items

      operator $ as pure
        <- rtn as String: $this.items

      default operator ?

  defines program

    CrashFilesDemo()
      stdout <- Stdout()

      stack <- Stack()
      stack.push("first")
      stack.push("second")
      stack.push("third")
      stdout.println(`Stack: ${stack}`)

      popped <- stack.pop()
      stdout.println(`Popped: ${popped}`)
      stdout.println(`Peek: ${stack.peek()}`)
      stdout.println(`Size: ${stack.size()}`)
Other ways to ask this
  • What is the ./fuzz-crashes/ directory?
  • How do I manage EK9 fuzzer output files?
  • What options control EK9 fuzz output locations?

Coming from another language?

Java: Jazzer saves crashes to a corpus directory, no structured manifest. Python: Atheris writes to crash-* files in current directory. Rust: cargo-fuzz saves to fuzz/artifacts/<target>/. Go: go test -fuzz saves to testdata/fuzz/<FuzzTestName>/. EK9: crash files in ./fuzz-crashes/, generated tests to specified file, mutations to specified directory with manifest.txt, all controllable via -overwrite, -seed, -n.

Keywords: crash, directory, files, fuzz-crashes, location, manifest, mutation, output, overwrite, reproducible, seed, variant