Where do crash files and generated tests go in EK9?

← Fuzzing and Mutation Testing · Ref: Q757

EK9 fuzzing tools write output to predictable locations with options to control overwriting and reproducibility.

CRASH FILES (-fuzz)
Location: ./fuzz-crashes/
When the compiler crashes on a generated file, the minimal reproduction case is saved here. Each file has a descriptive name including the phase that crashed and a hash of the stack trace.

GENERATED TESTS (-fuzztest)

  ek9 -fuzztest source.ek9 output.ek9

The second argument is the output file path. By default, the command errors if the output file already exists. Use -overwrite to replace it.

MUTATION VARIANTS (-fuzzmutate)

  ek9 -fuzzmutate source.ek9 mutations/

The second argument is the output directory. Contains:

  manifest.txt: lists each variant with mutation category and location.
  variant_001.ek9, variant_002.ek9, ...: individual mutated files.

By default, errors if the directory already exists. Use -overwrite to replace.

CONTROLLING OUTPUT

  -overwrite     Allow replacing existing output file or directory.
  -seed <n>      Reproducible generation (same seed produces same output).
  -n <n>         Maximum number of candidates (-fuzztest default 200, -fuzzmutate default 100).

CI WORKFLOW

In CI, use -overwrite so that each build produces fresh output:

  ek9 -fuzztest source.ek9 generated-tests.ek9 -overwrite
  ek9 -fuzzmutate source.ek9 mutations/ -overwrite -test tests.ek9

See Q749 for fuzzing overview. See Q750 for running the fuzzer. See Q753 for mutation testing. See Q754 for test generation. See Q2 for compile and run.

Example

defines module qa.fuzzingandmutation.crashfiles

  <?-
    Demonstrates code patterns that might produce crash
    files when fuzzed, and test output when generated.
    The output management options ensure CI repeatability.
  -?>

  defines class

    Stack
      items as List of String: List() of String

      push()
        -> item as String
        this.items += item

      pop()
        <- popped as String: String()

        count <- length this.items
        zero <- 0
        emptyDefault <- String()
        if count > zero
          lastIndex <- count - 1
          topItem <- this.items.getOrDefault(lastIndex, emptyDefault)
          if topItem?
            popped: topItem
            this.items -= topItem

      peek() as pure
        <- topItem as String: String()

        count <- length this.items
        zero <- 0
        emptyDefault <- String()
        if count > zero
          lastIndex <- count - 1
          topItem :=? this.items.getOrDefault(lastIndex, emptyDefault)

      size() as pure
        <- count as Integer: length this.items

      operator $ as pure
        <- rtn as String: $this.items

      default operator ?

  defines program

    CrashFilesDemo()
      stdout <- Stdout()

      stack <- Stack()
      stack.push("first")
      stack.push("second")
      stack.push("third")
      stdout.println(`Stack: ${stack}`)

      popped <- stack.pop()
      stdout.println(`Popped: ${popped}`)
      stdout.println(`Peek: ${stack.peek()}`)
      stdout.println(`Size: ${stack.size()}`)
Other ways to ask this
  • What is the ./fuzz-crashes/ directory?
  • How do I manage EK9 fuzzer output files?
  • What options control EK9 fuzz output locations?

Coming from another language?

Java: Jazzer saves crashes to a corpus directory, no structured manifest. Python: Atheris writes to crash-* files in current directory. Rust: cargo-fuzz saves to fuzz/artifacts/<target>/. Go: go test -fuzz saves to testdata/fuzz/<FuzzTestName>/. EK9: crash files in ./fuzz-crashes/, generated tests to specified file, mutations to specified directory with manifest.txt, all controllable via -overwrite, -seed, -n.

Keywords: seed, crash, manifest, reproducible, output, variant, overwrite, mutation, fuzz-crashes, files, directory, location