What is the input-variation (input-variety) metric in EK9?

← Code Quality · Ref: Q1362

The input-variation metric scores whether tests drive a reasonable VARIETY of input value-classes through each public construct. It is a floor detector, not a correctness certificate.

CODE COVERAGE IS NOT CORRECTNESS

EK9 also has code coverage built into the compiler and the -t test runner (the E83001 publish gate). But 100% code coverage only means every line and branch executed at least once during the tests - it does NOT prove the function is functionally correct, and it does NOT mean the inputs were varied. You can reach 100% coverage on multiply(Integer, Integer) with a single multiply(2, 2) call: every line runs, coverage is green, yet negatives, zero, overflow (min/max) and unset were never tried. Coverage answers 'did the code run?'; input-variety answers 'across how many meaningfully-different input classes did it run?'. The two are complementary, and neither certifies correctness - even full coverage AND full variety cannot prove that an assertion's expected value is actually right (that is what mutation testing, EK9's eventual stronger gate, targets). Input-variety exists to catch the common blind spot: green coverage masking a thin, low-variety test suite - the classic multiply(2, 2) case.

VALUE-CLASSES

Each parameter type admits a set of value-classes. Integer has 9 (unset, zero, negative, positive, typical, min, max, near-min, near-max), Boolean 3 (true/false/unset), String 7 (heuristic), a user enumeration = its members + unset. These are derived structurally from the type, no literals.

TWO FIGURES PER CALLABLE

FLOOR (the headline) = the 1-wise 'each-choice' count = the sum of each input's class-count. It asks: was each value-class of each input exercised at least once? For combine(Integer, Integer) the floor is 9 + 9 = 18 (reachable by about 9 well-chosen tests). This is the number the optional publish gate is built around.
CEILING (informational) = the pairwise opportunity = the sum over input pairs of classCount x classCount. For combine it is 9 x 9 = 81. It is an upper bound, never a test target - good 1-wise testing only reaches around 11% of it.

RECORDS DECOMPOSE TO FIELDS

A record is transparent, so it contributes its fields as separate leaf dimensions, never an atomic product. A Point{x, y} used by an operator is scored on its x and y leaves - so packaging fields into a record does not change the number, and Point.<=>(Point) is floor 36 / ceiling 486, not 81 x 81.

RECEIVER COUNTS

An instance method is scored against its RECEIVER's state too (the receiver is an implicit input), so a no-arg pop() or isSet() is measured against the states its object can be in - not treated as having no inputs.

PER-TYPE ROLLUP

A class/record/component/trait also shows a rollup summing its whole surface (constructors + methods), so you see how well the type as a whole is variety-tested, not just one member.

WHERE TO SEE IT

The MCP tool ek9_query_complexity in inputVariety mode lists every callable's floor and ceiling plus the per-type rollup; the IDE complexity hover shows the same figures per construct. A low floor means under-tested (or the type over-partitions its inputs); it never proves correctness. See Q811 for built-in code coverage, Q312 for the complexity metrics, and Q311 for the quality checks catalog.

Example

defines module qa.codequality.inputvariety

  defines function

    <?-
      Two Integer parameters admit 9 value-classes each, so the input-variation
      floor is 9 + 9 = 18 (exercise each class of each input once) and the pairwise
      ceiling is 9 x 9 = 81 (the informational upper bound).
    -?>
    combine() as pure
      ->
        left as Integer
        right as Integer
      <- rtn as Integer: left + right

  defines record

    <?-
      A record decomposes into its field leaves (x, y), so it is scored on its
      fields rather than as an atomic product - packaging does not change the number.
    -?>
    Point
      x Integer?
      y Integer?

      default private Point()

      Point()
        ->
          initX as Integer
          initY as Integer
        this.x: Integer(initX)
        this.y: Integer(initY)

      operator $ as pure
        <- rtn as String: `${x},${y}`

      default operator ?

  defines program

    InputVarietyDemo()
      stdout <- Stdout()

      total <- combine(2, 3)
      stdout.println(`${total}`)

      p <- Point(1, 2)
      stdout.println(`${p}`)

Coming from another language?

Combinatorial/pairwise testing (PICT, ACTS) and mutation testing (PIT for Java, Stryker for JS) all measure input adequacy but as separate, optional external tools run in CI. SonarQube reports coverage but not input-class variety. EK9 builds a cheap, always-on input-variety floor into the compiler alongside the other quality metrics, with mutation testing as the eventual stronger successor.

Keywords: value-class, quality, equivalence, ceiling, floor, pairwise, testing, input-variation, receiver, coverage, variety, metric, mutation, input-variety, boundary, correctness, each-choice