How do I group and aggregate collection data in EK9?

← Collections and Data Structures · Ref: Q124

EK9 stream pipelines support grouping items by a key extractor function. After grouping, you can filter, map, and flatten the groups.

GROUP BY KEY EXTRACTOR

Group by uses a function that extracts a key from each element:

  cat items | group by extractCategory | collect as List of List of Item

Elements with the same key value are collected into the same inner list. The input must be sorted by the same key first.

FILTER WITH (GROUPS)

Keep groups matching a predicate on each group (List of T):

  | filter with hasEnoughItems

The predicate receives the entire group (a List), not individual elements.

GROUP THEN MAP

Transform each group using a mapping function:

  | map by processGroup

The function receives a List of T and returns a transformed result.

FULL PIPELINE PATTERN

The complete group-process-flatten pattern:

  cat items
    | sort by keyExtractor
    | group by keyExtractor
    | filter with filterPredicate
    | map by groupTransformer
    | flatten
    > stdout

Sort first to ensure correct grouping. Group, filter, transform, then flatten back to a flat stream.

See Q89 for basic stream pipelines. See Q120 for sorting before group. See Q122 for collect as aggregation. See Q123 for flatten after group. See Q125 for head/tail/skip to limit results.

Example

defines module qa.collections.groupaggregate

  defines class

    Sale
      region as String?
      amount as Float?

      default private Sale() as pure

      Sale() as pure
        ->
          region as String
          amount as Float
        this.region :=? region
        this.amount :=? amount

      region() as pure
        <- rtn as String: String(region)

      amount() as pure
        <- rtn as Float: Float(amount)

      operator $ as pure
        <- rtn as String: `${region}:${amount}`

      operator <=> as pure
        -> other as Sale
        <- rtn as Integer: region <=> other.region

      operator #? as pure
        <- rtn as Integer: #?region

      override operator ? as pure
        <- rtn as Boolean: region? and amount?

  defines function

    regionKey() as pure
      -> sale as Sale
      <- rtn as String: sale.region()

    comparingRegion() as pure
      ->
        t1 as Sale
        t2 as Sale
      <-
        rtn as Integer: t1.region() <=> t2.region()

    hasTwoOrMore() as pure
      -> group as List of Sale
      <- rtn <- Boolean()
      expectedGroups <- 2
      rtn: length group >= expectedGroups

  defines program

    GroupAggregateDemo()
      stdout <- Stdout()

      sales <- [
        Sale("North", 100.0),
        Sale("South", 200.0),
        Sale("North", 150.0),
        Sale("East", 300.0),
        Sale("South", 175.0),
        Sale("North", 125.0)
        ]

      // === GROUP BY REGION ===

      grouped <- cat sales
        | sort by comparingRegion
        | group by regionKey
        | collect as List of List of Sale
      stdout.println(`Grouped: ${grouped}`)

      // === FILTER GROUPS WITH 2+ SALES ===

      filtered <- cat sales
        | sort by comparingRegion
        | group by regionKey
        | filter with hasTwoOrMore
        | collect as List of List of Sale
      stdout.println(`Filtered: ${filtered}`)

      // === FULL PIPELINE: GROUP, FILTER, FLATTEN ===

      cat sales
        | sort by comparingRegion
        | group by regionKey
        | filter with hasTwoOrMore
        | flatten
        > stdout

Common mistakes

E50060 — EK9 uses short method names without 'get' prefix. Sale has region() and amount() not getRegion() and getAmount(). Triggers E50060 — method not resolved. See ek9 -h E50060 for details.

Incorrect:

sale.getRegion()

Correct:

sale.region()

E50060 — List has no toString() method. Use the $ prefix operator or string interpolation for string conversion. See ek9 -h E50060 for details.

Incorrect:

stdout.println(grouped.toString())

Correct:

stdout.println(`Grouped: ${grouped}`)

E07830 — Group produces List of T from a stream of T — each group is a list. So the collection must be List of List of T, not List of T. A List of String cannot receive List of String items via the pipe operator. See ek9 -h E07830 for details.

Incorrect:

| collect as List of Sale

Correct:

| collect as List of List of Sale

E50030 — The group function's parameter type must match the stream's element type. Using a function that accepts Date on a String stream triggers E50030 because the types are incompatible. See ek9 -h E50030 for details.

Incorrect:

| group by hasTwoOrMore

Correct:

| group by regionKey

E06300 — The group by function must accept exactly one argument — the current stream element. A function with two parameters cannot be used for grouping. See ek9 -h E06300 for details.

Incorrect:

| group by comparingRegion
        | filter with hasTwoOrMore

Correct:

| group by regionKey
        | filter with hasTwoOrMore

E07470 — The sort by comparator must accept exactly two parameters of the element type and return Integer. A single-parameter function cannot compare two elements. Sort is required before group to ensure correct grouping. See ek9 -h E07470 for details.

Incorrect:

| sort by regionKey

Correct:

| sort by comparingRegion

E50060 — List has no toString() method. Use the $ prefix operator or string interpolation for string conversion. See ek9 -h E50060 for details.

Incorrect:

stdout.println(filtered.toString())

Correct:

stdout.println(`Filtered: ${filtered}`)

Coming from another language?

Java: Collectors.groupingBy() with downstream collectors. Python: itertools.groupby() (requires pre-sort). JavaScript: manual reduce into Map or lodash groupBy. Rust: itertools group_by() (requires pre-sort). Go: manual loop with map. EK9: cat | sort by key | group by key | filter with | map by | flatten pipeline, requires pre-sort like Python/Rust itertools.

Keywords: partition, group, key, flatten, extractor, category, pipeline, aggregate, list, collection, sort, map, groupingBy, filter, groupBy, data-structure