E10040: 'group' requires a preceding 'sort' with no type-changing operations between them

Category: E10xxx IR generation · Reported in phase: FULL_RESOLUTION · Classification: SORT_REQUIRED_BEFORE_GROUP · All error codes

What went wrong

The 'group' operation uses change-detection to collect adjacent items with the same hashcode into sublists. This requires items to be sorted so that items with the same group key are adjacent in the stream. Without a preceding 'sort', items with the same key may be scattered, producing multiple incomplete groups instead of one complete group.

Worked examples

Compilable examples demonstrating this error and the correct fix:

  • ek9 -q 891 (Why must I sort before grouping in an EK9 stream pipeline?)

[TYPE-CHANGING OPERATIONS INVALIDATE SORT] A 'sort' is only valid for a subsequent 'group' if no type-changing operations occur between them. Operations like 'map', 'flatten', 'split', and 'group' itself change the stream type, invalidating any earlier sort. Sorting R1 items, then mapping them to R2, then grouping is this error: the sort was on R1, not on the R2 being grouped. Add a sort after the type change ('| sort by comparingR2' before the '| group'). A second 'group' straight after a first is the same error.

[TYPE-PRESERVING OPERATIONS ARE FINE] Operations like 'filter', 'reject', 'head', 'tail', 'skip', 'uniq', and 'tee' preserve the stream type and do not invalidate a preceding sort: sorting R1 items, filtering them and then grouping is fine. The [EXAMPLE] section below shows every one of these cases.

How to fix it

  1. Add '| sort' or '| sort by comparator' before '| group' in your pipeline
  2. Ensure no type-changing operations (map, flatten, split) occur between the sort and the group
  3. If a type change is needed, add another sort after the transformation

See also

  • E10040: group requires preceding sort (this error)
  • E10020: void in stream pipeline
  • E07840: missing iterator for cat

Examples

Every example is compiled by the EK9 compiler: the first fails exactly as captioned, the others fix it.

Fails with E10040: 'group' requires a preceding 'sort' with no type-changing operations between them

#!ek9
defines module error.e10040.bad

  defines record

    R1 as open
      name as String?

      default private R1()

      R1()
        -> text as String
        this.name: text

      //Equatable (not orderable) sameness; its '#?' goes with it.
      default operator ==

      default operator ?

      default operator $

      default operator #?

    R2
      label as String?

      default private R2()

      R2()
        -> text as String
        this.label: text

      //Equatable (not orderable) sameness; its '#?' goes with it.
      default operator ==

      default operator ?

      default operator $

      default operator #?

  defines function

    <?-
      Maps R1 to R2 - this is a type-changing function.
    -?>
    MapR1toR2()
      -> item as R1
      <- rtn as R2: R2($item)

    <?-
      Filter function that keeps items with non-empty names.
    -?>
    hasContent()
      -> item as R1
      <- rtn as Boolean: item?

    comparingR1() as pure
      ->
        r1 as R1
        r2 as R1
      <- rtn as Integer: $r1 <=> $r2

    comparingR2() as pure
      ->
        r1 as R2
        r2 as R2
      <- rtn as Integer: $r1 <=> $r2

  defines function

    <?-
      Group without any sort - the simplest invalid case.
    -?>
    GroupWithNoSort()
      collection <- List() of List of String

      cat ["A", "Z", "Z", "P", "B"] | group > collection

      require collection?

    <?-
      Sort followed by map changes the type, so the sort is invalidated.
      The group sees R2 but the sort was on R1.
    -?>
    GroupAfterMapInvalidatesSort()
      collection <- List() of List of R2

      cat [R1("A"), R1("Z"), R1("P")] | sort by comparingR1 | map with MapR1toR2 | group > collection

      require collection?

    <?-
      First group changes type from T to List of T, invalidating the sort for a second group.
    -?>
    SecondGroupAfterFirstGroupInvalidatesSort()
      collection <- List() of List of List of String

      cat ["A", "Z", "Z", "P"] | sort | group | group > collection

      require collection?

    <?-
      Filter is type-preserving, so sort remains valid through filter.
      This should compile successfully.
    -?>
    GroupAfterFilterPreservesSort()
      collection <- List() of List of R1

      cat [R1("A"), R1("Z"), R1("P")] | sort by comparingR1 | filter by hasContent | group > collection

      require collection?

    <?-
      Valid: sort immediately before group.
    -?>
    GroupWithImmediateSort()
      collection <- List() of List of String

      cat ["A", "Z", "Z", "P", "B"] | sort | group > collection

      require collection?

    <?-
      Valid: sort with comparator before group.
    -?>
    GroupWithSortByComparator()
      collection <- List() of List of R1

      cat [R1("A"), R1("Z"), R1("P")] | sort by comparingR1 | group > collection

      require collection?

Fixed: Sort immediately before group (a type-preserving filter between is fine)

#!ek9
defines module error.e10040.good

  defines function

    groupSorted()
      <- rtn as List of List of String: List()
      rtn: cat ["A", "Z", "Z", "P", "B"] | sort | group | collect as List of List of String

Questions and answers

These Q&A entries show E10040 and its fix: