How do I read the EK9 fuzz HTML dashboard?

← Fuzzing and Mutation Testing · Ref: Q752

The -fuzz6 flag generates an interactive HTML dashboard at fuzz-report/index.html . Summary cards (including the programs-per-second rate) sit at the top, followed by sections on the pipeline, error coverage, the generation strands and crashes.

PHASE PENETRATION FUNNEL AND PHASE DISTRIBUTION

The funnel shows what fraction of the generated programs survive to each compilation phase (PARSING, SYMBOL_DEFINITION, TYPE_HIERARCHY_CHECKS, PRE_IR_CHECKS, etc.); the distribution shows where programs stop. A healthy fuzzer produces files that exercise all phases, not just the parser.

ERROR CODE COVERAGE

A grid with one cell per compiler error code. Green cells were triggered (darker means more frequent), grey cells never were. You can search the grid and filter it to triggered or untriggered codes. An Untriggered Error Code Analysis section groups the missing codes by phase, which reveals gaps in the fuzzer's grammar coverage.

CONVERGENCE AND STRANDS

Error Coverage Convergence shows how error coverage grows over the run. The Error-Strand Discovery Heatmap, the strand comparison charts and the Adaptive Weight Timeline show which generation strands found which errors and how their weights moved.

CONSTRUCT COVERAGE

Which EK9 constructs (classes, records, traits, functions, services...) the generated programs exercised.

CRASH AND LIMITATION BREAKDOWN

Summary cards for compiler crashes and for known limitations (expected failures from unimplemented features), with the crash rate. The crash files themselves are saved in ./fuzz-crashes/.

READING THE DASHBOARD

- All error codes triggered = good fuzzer coverage.
- Files reaching deep phases = grammar generator working well.
- Zero crashes after long runs = compiler is robust.
- Surviving error code gaps = improve generator for those constructs.

See Q749 for fuzzing overview. See Q751 for output formats. See Q627 for profiling dashboard (similar HTML pattern). See Q321 for quality report dashboard.

Example

defines module qa.fuzzingandmutation.htmldashboard

  <?-
    Code with multiple error paths that produce
    a rich distribution in the fuzz heatmap.
    Different branches exercise different compiler
    phases and error detection logic.
  -?>

  defines trait

    Measurable
      measurement() as pure
        <- rtn as Float: 0.0

      label() as pure
        <- rtn as String: String()

  defines class

    Sensor as open
      sensorName as String: String()
      currentReading as Float: 0.0

      Sensor()
        ->
          initialName as String
        this.sensorName :=? initialName

      reading() as pure
        <- rtn as Float: this.currentReading

      updateReading()
        -> newReading as Float
        this.currentReading: newReading

      operator $ as pure
        <- rtn as String: `${this.sensorName}: ${this.currentReading}`

      default operator ?

  defines function

    summariseSensors()
      -> sensors as List of Sensor
      <- summary as String: "No sensors"

      count <- length sensors
      zero <- 0
      if count > zero
        total <- 0.0
        for sensor in sensors
          total: total + sensor.reading()
        avg <- total / #^count
        summary: `${count} sensors, avg ${avg}`

  defines program

    FuzzHtmlDashboardDemo()
      stdout <- Stdout()

      alpha <- Sensor("alpha")
      beta <- Sensor("beta")
      gamma <- Sensor("gamma")

      alpha.updateReading(23.5)
      beta.updateReading(19.8)
      gamma.updateReading(27.1)

      sensors <- [alpha, beta, gamma]
      for sensor in sensors
        stdout.println($sensor)

      stdout.println(summariseSensors(sensors))

Coming from another language?

Java: Jazzer provides basic crash reports, no HTML dashboard. Python: Atheris writes crash files only, visualisation requires custom tools. Rust: cargo-fuzz has no built-in dashboard. Go: go test -fuzz writes to testdata, no visualisation. EK9: built-in interactive HTML dashboard with an error code coverage grid, charts and crash counts generated by -fuzz6.

Keywords: throughput, report, dashboard, fuzz, error, crash, html, heatmap, phase, statistics, code, visualisation