Remove duplicate integers from a list using sort and uniq.

Sort first, then uniq removes consecutive duplicates:

  cat items | sort | uniq > stdout

Sort ensures equal items are adjacent so uniq can remove them. See Q989, Q891.

Example

defines module qa.streams.streamuniqgroup

  defines program

    StreamUniqGroupDemo()
      stdout <- Stdout()

      items <- [3, 1, 2, 1, 3, 2, 1]

      //Sort then uniq to deduplicate
      stdout.println("Sorted unique:")
      cat items | sort | uniq > stdout

      //Sort, group, then flatten
      stdout.println("Sort group flatten:")
      cat items | sort | group | flatten > stdout

Common mistakes

E01010 — EK9 uses 'uniq' (not 'distinct'). Sort first to ensure equal items are adjacent.

Incorrect:

      cat items | distinct > stdout

Correct:

      cat items | sort | uniq > stdout

Other ways to ask this

I have a list with repeated values and need to keep only unique items
In Python I'd use set() for dedup. Write the EK9 stream deduplication
Given [3,1,2,1,3,2], produce a sorted list with no duplicates
Deduplicate a list using a sort | uniq pipeline

Coming from another language?

Java: stream().distinct().sorted(). Python: sorted(set(items)). Rust: dedup() after sort. EK9: sort | uniq — two-stage pipeline.

Keywords: distinct, unique, group, uniq, sort, deduplicate, flatten