How do I group tests and control execution order in EK9?

← Testing · Ref: Q806

EK9 test grouping controls whether tests run in parallel or sequentially, and lets you run subsets.

UNGROUPED TESTS RUN IN PARALLEL

Tests without a group name run in parallel:

  @Test
  TestA()
    assert 1 + 1 == 2
  @Test
  TestB()
    assert 2 + 2 == 4

TestA and TestB may run simultaneously on different threads.

GROUPED TESTS RUN SEQUENTIALLY

Tests in the same group run one after another:

  @Test: "database"
  SetupDbTest()
    // runs first
  @Test: "database"
  QueryDbTest()
    // runs after SetupDbTest

This guarantees ordering within the group. Different groups still run in parallel.

RUNNING A SPECIFIC GROUP

Use -tg to run only tests in a named group:

  ek9 -tg database myApp.ek9

This skips all tests not in the "database" group.

LISTING TESTS WITHOUT RUNNING

Use -tL to discover tests without executing them:

  ek9 -tL myApp.ek9

Shows all test programs and their groups. Combine with -tg:

  ek9 -tL -tg database myApp.ek9

WHEN TO USE GROUPS

- Tests that share state (database, files): group them sequentially
- Tests that modify shared resources: group to avoid race conditions
- Fast independent tests: leave ungrouped for parallel speed

See Q155 for basic testing. See Q207 for output formats. See Q804 for why built-in.

Example

defines module qa.testdeep.grouping

  defines function

    initializeState() as pure
      <- rtn as String: "ready"

    processItem() as pure
      -> item as String
      <- rtn as String: `processed: ${item}`

  defines program

    // === UNGROUPED: RUN IN PARALLEL ===

    @Test
    FastCheckA()
      assert 1 + 1 == 2

    @Test
    FastCheckB()
      assert 2 * 3 == 6

    // === GROUPED: RUN SEQUENTIALLY ===

    @Test: "workflow"
    WorkflowStepOne()
      state <- initializeState()
      assert state == "ready"

    @Test: "workflow"
    WorkflowStepTwo()
      result <- processItem("data")
      assert result == "processed: data"

    // ek9 -tg workflow myApp.ek9  <- runs only "workflow" group
    // ek9 -tL myApp.ek9           <- lists all tests and groups

Coming from another language?

Java: JUnit @Tag for filtering, @TestMethodOrder for ordering, @Execution(SAME_THREAD) for sequential. Python: pytest -k for filtering, pytest-ordering for order control. Rust: #[test] runs in parallel by default, --test-threads=1 for sequential. Go: t.Run() subtests, -run regex filtering. EK9: @Test: "group" for sequential grouping, -tg for filtering, ungrouped tests run in parallel.

Keywords: group, execution, parallel, subset, sequential, filter, tg, list, tL, order