Quality Built In

Testing as You Go ended on a promise: coverage is the one quality measure that gates, and the variety figure is a report that shows where your tests are thin. Components said the compiler "keeps score" of coupling and cohesion. Testing the Compiler Trusts then showed coverage gating a run and input variety measuring beside it. This chapter is the rest of the scorecard, and the argument that ties it together: in EK9 the compiler has been grading you since your first function, and nearly every grade is a compile error with a figure, a limit and no way round it.¹

If you use SonarQube, Checkstyle, PMD or ESLint you know most of these measures already, down to the name; EK9's cognitive complexity is Sonar's idea. What differs is the consequence. A linter reports, a dashboard goes amber, and a deadline turns the finding into a ticket. Here the same finding stops the build.

Rejected, with a figure

Here is a function that summarises a list of orders: how many lines fall in each price band, how many are bulk, how many are empty, how many orders are priority. It does the whole job in one place, and every decision nests inside the one before it:

summarise()
  -> orders as List of Order
  <- rtn as String?

  expensive <- 0
  moderate <- 0
  cheap <- 0
  bulkLines <- 0
  emptyLines <- 0
  priorityIds <- List() of String

  for order in orders
    if order.id?
      for line in order.lines
        if line.quantity > 0
          if line.price > HIGH_PRICE
            expensive++
          else if line.price > MID_PRICE
            moderate++
          else
            cheap++
      for item in order.lines
        if item.sku?
          if item.quantity >= BULK
            bulkLines++
          else if item.quantity == 0
            emptyLines++
      if order.priority
        if length of order.lines > 0
          priorityIds += order.id

  rtn: `expensive ${expensive}, moderate ${moderate}, cheap ${cheap}, bulk ${bulkLines}, empty ${emptyLines}, priority ${length of priorityIds}`

It reads like the code most of us write on a first pass, and it is rejected:

EK9     : Error   : E11021: 'summarise' on line 47 position 4: Cognitive complexity of 44 exceeds maximum of 35: cognitive complexity exceeds limit - extract functions
             See: ek9 -h E11021
           forAI: qualityMetrics.cognitiveComplexity

The message gives the figure and the limit. Cognitive complexity is scored per control structure and multiplied by the nesting depth it sits at, and a <- guard adds a penalty of one (ek9 -h complexity): a loop inside an if inside a loop costs far more than three flat ones, which is the intuition behind the metric. The limit is 35 and this function scores 44. There is no @SuppressWarnings to put above it, no threshold to raise in a file, and no way to compile it today and fix it next sprint. It is the only error in that file; the two programs share their records and their OrderReport program, and differ in summarise.

The fix is the one the limit steers towards. Each decision becomes a one-line pure function with a name, and a pipeline says how they combine:

bandMatcher() as pure abstract
  -> band as String
  <- rtn as Boolean?

hasId() as pure
  -> order as Order
  <- rtn as Boolean: order.id?

linesOf() as pure
  -> order as Order
  <- rtn as List of OrderLine: order.lines

hasQuantity() as pure
  -> line as OrderLine
  <- rtn as Boolean: line.quantity > 0

priceBand() as pure
  -> line as OrderLine
  <- rtn as String: line.price > HIGH_PRICE <- EXPENSIVE : (line.price > MID_PRICE <- MODERATE : CHEAP)

isBulk() as pure
  -> line as OrderLine
  <- rtn as Boolean: line.quantity >= BULK

hasNothing() as pure
  -> line as OrderLine
  <- rtn as Boolean: line.quantity == 0

isPriority() as pure
  -> order as Order
  <- rtn as Boolean: order.priority and length of order.lines > 0

bandCount() as pure
  ->
    bands as List of String
    wanted as String
  <- rtn as Integer: 0

  sameBand <- (wanted) is bandMatcher as pure function
    rtn :=? band == wanted
  found <- cat bands | filter by sameBand | collect as List of String
  rtn: length of found

summarise() as pure
  -> orders as List of Order
  <- rtn as String?

  validOrders <- cat orders | filter by hasId | collect as List of Order
  lines <- cat validOrders | map with linesOf | flatten | collect as List of OrderLine
  bands <- cat lines | filter by hasQuantity | map with priceBand | collect as List of String
  bulkLines <- cat lines | filter by isBulk | collect as List of OrderLine
  emptyLines <- cat lines | filter by hasNothing | collect as List of OrderLine
  priorityOrders <- cat validOrders | filter by isPriority | collect as List of Order

  rtn: `expensive ${bandCount(bands, EXPENSIVE)}, moderate ${bandCount(bands, MODERATE)}, cheap ${bandCount(bands, CHEAP)}, bulk ${length of bulkLines}, empty ${length of emptyLines}, priority ${length of priorityOrders}`

Two details are worth reading. flatten turns the list of lines inside each order into one list of lines, which is what removed the second for. And bandCount takes a captured value, wanted, into a dynamic function (Dynamic Functions and Classes), so that one function counts any band. None of the functions in the listing nests a control structure, which is the shape the limits push you towards. The records and the OrderReport program are the same as in the rejected file, and the program prints the line the summary builds:

expensive 1, moderate 2, cheap 1, bulk 2, empty 1, priority 1

Stream Pipelines in Depth showed the same effect on a loop that nested seven deep: it printed the cyclomatic, cognitive and nesting figures side by side, and the loop form stopped compiling at E11011. Those are three of five limits on a function, and all five are errors:

Measure	Limit	Code
Cyclomatic complexity	45 per function, 500 per class	E11010
Cognitive complexity	35	E11021
Nesting depth	6	E11011
Nested `?:`, `<?` and `>?`	15	E11013
Share of the cyclomatic limit multiplied by share of the statement limit	0.50	E11020

The last row is the unusual one. Each of the others can be met by a function that is large and flat, or small and tangled. E11020 multiplies the share of the cyclomatic limit by the share of the statement limit (50 statements for an operator, 100 for a method, 150 for a function), so a function that is moderately both is refused although neither limit alone has been reached. Every figure above is ek9 -h complexity, and the compiler's error text prints the exact figure for the function it refused.

The architecture layer

Most languages farm the next group of measures out to a second tool, or to a design review. They are about the shape of a type and a module, not of a function, and EK9 checks them in the same pass, with the same lack of mercy. Each of the examples that follow is a program that compiles in every other respect.

Cohesion. A class whose methods split into groups that share no fields is two classes pretending to be one. The check is LCOM4, the count of connected groups in the graph of which method touches which field; one is ideal. The limit is 8 for a class and 10 for a component. Records, traits, functions and services are exempt by design, because none of them has a graph of methods and fields to split; a service has no fields at all. A dashboard that counts nine unrelated things has nine groups:

EK9     : Error   : E11014: 'Dashboard' on line 6 position 4: LCOM4 of 9 exceeds maximum of 8 for CLASS: low cohesion - class has disconnected method groups, consider splitting
             See: ek9 -h E11014
           forAI: qualityMetrics.cohesion

The class has nine counters and nine one-line methods, each touching its own counter, and an ordinary default operator ?. Two methods are joined when they touch the same field or one calls the other, on the object itself: a call or a field read on another instance of the class, other.total(), joins nothing. An abstract method has no body, so it is never a group of its own, though the methods that call it are joined through it. Constructors and the operators that work on the whole value - ?, ==, <=>, #?, $, :=: and their kind - are left out of the graph, generated or written by hand, because they read every field by definition: counting them would join every group and make any class look cohesive. What is left is the class's behaviour, and that is what the check measures.

Coupling. Efferent coupling, Ce, counts the other types a construct depends on: field, parameter and return types, supers and implemented traits, and variables captured into a dynamic function or class. The built-in types are not counted. The limit is 12 for a class, 15 for a component or service, and 8 for a record, trait or function (E11015). A class that holds thirteen domain types as fields is refused:

EK9     : Error   : E11015: 'OrderDesk' on line 60 position 4: Efferent coupling of 13 exceeds maximum of 12 for CLASS: excessive efferent coupling - depends on too many types
             See: ek9 -h E11015
           forAI: qualityMetrics.coupling

The module has a coupling limit too, E11016: ten external modules, counted by module and not by type, so five types from one module count once and the usual cure is a wrapper module that gathers a family of dependencies behind one. The module also has a cohesion check, E11017, for a module of at least sixty constructs that falls apart into thirty or more groups that share no types; at that size it is a utility module that wants splitting into one module per purpose, and it needs more constructs than a book page can hold, so there is no example.

Inheritance depth. The ceilings are 4 levels for a class, trait or component, 3 for a function and 2 for a record (E11019), and the message names the alternative, delegation with by:

EK9     : Error   : E11019: 'Director' on line 18 position 4: Inheritance depth of 3 exceeds maximum of 2 for RECORD: excessive inheritance depth - consider delegation with 'by'
             See: ek9 -h E11019
           forAI: qualityMetrics.inheritanceDepth

Shape. Two checks look at what a class is. A class made only of service fields, objects that do things and no data, is a component in disguise (E11022, four service fields and no data fields), and the message says defines component, the construct Dependency Injection and Applications builds on. The limits for the related shapes are in ek9 -h quality: a class that overrides most of what it extends wants delegation (E11023), and one with three services and three data fields is doing two jobs (E11025).

EK9     : Error   : E11022: 'Notifier' on line 44 position 4: has 4 service fields (mailer, auditor, archiver, scheduler) and no data fields: class has component characteristics - use 'defines component'
             See: ek9 -h E11022

The last in this family is the data clump: the same four parameters repeated across three functions, which is a record that has not been written yet (E11053):

EK9     : Error   : E11053: 'book' on line 2 position 15: Data clump of 4+ shared parameters across 3 callables in 'book.quality.clump': [formatAddress, isComplete, labelFor] data clump detected - extract a record to group repeated parameters
             See: ek9 -h E11053

Extracting the record is the cure, and the three functions each take one argument:

defines function

  formatAddress() as pure
    -> address as Address
    <- rtn as String: `${address.street}, ${address.city}, ${address.state} ${address.zipCode}`

  isComplete() as pure
    -> address as Address
    <- rtn as Boolean: address.street? and address.city? and address.state? and address.zipCode?

  labelFor() as pure
    -> address as Address
    <- rtn as String: `${address.zipCode}: ${address.street} ${address.city} ${address.state}`

1 High Street, Leeds, West Yorkshire LS1 1AA
true
LS1 1AA: 1 High Street Leeds West Yorkshire

Naming, which Volume 1 introduced one rule at a time, belongs to the same family: a banned word such as temp or data is E11031, two names one edit apart are E11030, and an unsorted references list is E11026. They share the property that matters: none of them is a warning. ek9 -h quality ends its list with the statement that the compiler has "NO warnings -- only errors", and the program that gets through has passed all of them. Packaging asks for the same gate plus the 80% coverage bar before it will build (Packaging and Sharing Your Program).

What coverage cannot say

Coverage gates. It is also easy to meet with tests that ask only the one easy question, which is the weakness Testing the Compiler Trusts measured with input variety. The quality report goes one step further and joins that measurement to the complexity figures above. The result is HEX, hazard multiplied by exposure, a change-risk score from 0 to 100 where higher is worse (ek9 -h hex):

hazard is how much a change can break, from cognitive complexity and efferent coupling;
exposure is how little the tests would notice, from coverage and from asserted input variety.

The question it answers is one coverage cannot: if I change this, how much can break, and would anything notice? Here is a small function with some branches in it, and a first test:

defines constant
  LIGHT_LIMIT <- 500
  HEAVY_LIMIT <- 5000
  LIGHT_PRICE <- 300
  STANDARD_PRICE <- 450
  HEAVY_PRICE <- 900
  EXPRESS_SURCHARGE <- 250

defines function

  postage() as pure
    ->
      grams as Integer
      express as Boolean
    <- rtn as Integer: STANDARD_PRICE

    if grams <= LIGHT_LIMIT
      rtn: LIGHT_PRICE
    else if grams > HEAVY_LIMIT
      rtn: HEAVY_PRICE

    if express
      rtn: rtn + EXPRESS_SURCHARGE

@Test
LetterByStandardPost()
  assert postage(100, false) == 300

One test, the letter sent by standard post. Run it with ek9 -t -tC, and the gate stops it:

📊 Coverage Summary
  ✅ Methods:  100.0% (1/1)
  ⚠️ Lines:    60.0% (6/10)
  ❌ Branches: 33.3% (1/3)

  ⚠️ Overall: 60.00% (6/10 probes hit)

📦 By Module:
  ⚠️ book.quality.postage: 60.00% (6/10)

The run then names the file that holds the detail of what was missed, and ends:

❌ E83001: Code coverage 60.0% is below required threshold 80%
   Methods:  100.0% (1/1)
   Lines:    60.0% (6/10)
   Branches: 33.3% (1/3)

   Uncovered items:
     2 branch(es)

   Use -t4 or -t5 for detailed coverage report.
ek9: coverage below the required threshold (exit 12)

Coverage is the gate, and it is right to stop here: two of the three branches never ran. -t6 writes the HTML quality report, and the entry it holds for postage shows the figures this chapter has been naming, together. The text below is that entry with the markup removed and the lines wrapped:

postage()  ./postage.ek9:14
CC:6 Complexity — cyclomatic 6, cognitive 3, nesting depth 1. Cyclomatic counts the independent paths through the
    code; cognitive is how hard it is to follow; nesting is the deepest stack of control structures. Higher means
    harder to test and to change safely.
I:2/12 Input variety — 2 of 12 input value-classes driven (16.7%); 2 asserted on (16.7%). Coverage says the lines ran;
    this says across how many kinds of input, and how many of those a test actually checked. Green coverage over a low
    variety is thin testing.
H9·E83 Change risk (HEX 7) — hazard 9% (how much a change here can break, from complexity and coupling) × exposure 83%
    (how little the tests would notice). Two levers, always: make the code less dangerous, or make the tests notice
    more.

Hazard is 9: the function is small, with a cyclomatic complexity of 6 and a cognitive complexity of 3. Exposure is 83: of twelve input classes the parameters allow, two were driven, and the HEX score is the product, 7. It is a low score and the function is untested. The help text is blunt about this: "A LOW score means low change-RISK, NOT 'safe' or 'well tested'. A trivial construct scores low because there is little to go wrong, not because it is guarded." HEX is not a verdict on the tests. It estimates what a mistake in this function would cost and whether the tests would catch it.

Now give it six tests that between them take the branches and a range of weights, among them zero and a negative one (dev/tests.ek9 in the download):

📊 Coverage Summary
  ✅ Methods:  100.0% (1/1)
  ✅ Lines:    100.0% (10/10)
  ✅ Branches: 100.0% (3/3)

  ✅ Overall: 100.00% (10/10 probes hit)

📦 By Module:
  ✅ book.quality.postage: 100.00% (10/10)

Coverage is full, and the gate is satisfied. The entry from the same report:

postage()  ./postage.ek9:14
CC:6 Complexity — cyclomatic 6, cognitive 3, nesting depth 1. Cyclomatic counts the independent paths through the
    code; cognitive is how hard it is to follow; nesting is the deepest stack of control structures. Higher means
    harder to test and to change safely.
I:6/12 Input variety — 6 of 12 input value-classes driven (50.0%); 6 asserted on (50.0%). Coverage says the lines ran;
    this says across how many kinds of input, and how many of those a test actually checked. Green coverage over a low
    variety is thin testing.
H9·E50 Change risk (HEX 4) — hazard 9% (how much a change here can break, from complexity and coupling) × exposure 50%
    (how little the tests would notice). Two levers, always: make the code less dangerous, or make the tests notice
    more.

Exposure fell from 83 to 50 and HEX from 7 to 4. The reason it fell no further is the line -t5 prints under its coverage summary, the classes the tests drove:

Input-variety (numerator, measurement-only):
  book.quality.postage::postage                      driven 6 cell(s), asserted 6: p0: NEGATIVE,POSITIVE,TYPICAL,ZERO  p1: FALSE,TRUE

Every line ran, and six of the twelve input classes were driven. The floor is a static figure the compiler works out from the parameter types, and nothing enforces it. The sentence that follows the figures in the report states its authority exactly: HEX is advisory and is never a gate, and coverage gates (ek9 -h variety). If you read a score as a threshold you would be asking the compiler for something it does not claim to compute.

Three reading rules come from ek9 -h hex, and each guards a mistake that is easy to make:

Read the mean and the maximum together. A calm average over a dangerous outlier is the usual misreading; the maximum names the construct that will bite.
The cohorts differ. A construct whose parameter type is loose, a bare String, has no exact input space, so its floor is a guess and it is counted apart as approximate. The cure is to tighten the type, an enumeration or a constrained type, which moves it into the measured cohort and makes the input space explicit to the next reader. A construct with no parameters, or a compiler-written body, has no input space at all and is scored on coverage alone.
Afferent coupling is deliberately not in the score. How many callables depend on a construct is reported on its own, because multiplying blast radius by test adequacy would let a well-tested type that fifty things depend on look safe. High dependents with a low score means: change the insides freely, and do not change the signature.

A suite validated by expected-output files rather than assertions shows driven classes and no asserted ones, because only an assert opens the window in which a result is observed; Testing the Compiler Trusts made the same point. HEX's two levers are in the report's own sentence, and they are the only two: make the code less dangerous, or make the tests notice more.

The cost

You cannot wave any of this through. A function at 36 is refused as firmly as one at 360, so a long switch that is clear to read, and exactly the right shape for the job, will still have to be split. The limits are the compiler's, not yours: you cannot raise one for the module that is a special case, and the special case is real more often than anyone likes. Most of the checks measure a proxy, not the harm. LCOM4 counts groups of fields and methods, and a class can have one group and still be a muddle; a count of types is not a count of reasons to change. You will meet the first error just when you wanted to finish, and the instinct it provokes, to split the function until the number drops, can produce the thing the limit was written to avoid: five functions that are only meaningful together, named after their position in the original. A pipeline can be a worse read than the loop it replaced (Stream Pipelines in Depth says so).

A design note. A linter that can be turned off will be turned off, on the afternoon before a release, by someone who is right that the deadline matters. I do not want to win that argument with a better argument. I want the argument not to be available. So the limit lives in the compiler, there is no comment that disables it, and the figure in the message is the same for everyone who reads it, including the next person to inherit the code. It costs you the afternoon on which the limit is wrong. I think that is a good price for never again inheriting a function that nobody dared touch. - Steve

What it buys is a floor, not a ceiling. Code that compiles has passed every check in this chapter, so a reader meets no function above the cognitive limit and no class that depends on thirteen types, in this project or in a dependency that built. What it does not say is that the design is good, or that the tests ask the right questions: HEX estimates exposure and the compiler has no way to know what the right question was. The comment is still the one thing the compiler cannot check (Organising a Growing Codebase), and this chapter is the other side of that argument: everything that can be measured has been, and the result is an error.

If you skipped Volume 1, Testing as You Go introduces coverage and the 80% bar, and Components introduces coupling and cohesion. ↩