E82001: test execution timeout

What went wrong

The @Test program was still running when its time limit ran out, so it failed. Every @Test run (each case of a parameterised test too) has the same limit: EK9_TEST_TIMEOUT_MS milliseconds, 30000 (30 seconds) unless you set it. The test is stopped and the rest of the run goes on: a native test is killed with every process it started; the IDE's and an AI's test runs each have a JVM of their own, killed with everything it started; under 'ek9 -t' a JVM test's run is interrupted and abandoned, and 'ek9 -t' still ends when the other tests are done.

Worked examples

Compilable examples demonstrating this error and the correct fix:

• ek9 -q 953 (How do @Test programs work with command-line arguments...)

Example error

In the [EXAMPLE] below, 'pairsToGreet' takes guests two at a time 'while waiting <> 0', which never ends for an odd number of guests, so the test 'FiveGuestsMakeTwoPairs' fails at the limit (run with EK9_TEST_TIMEOUT_MS=1000, so 1000 ms) and the other tests pass.

How to fix it

1. Fix the loop or the blocking call that never ends
2. Do not wait on a real network or file in a test: give it a stand-in that answers at once
3. A test that is slow but correct: split it, or raise the limit - export EK9_TEST_TIMEOUT_MS=60000, or put EK9_TEST_TIMEOUT_MS=60000 in ~/.ek9/ek9.env (the IDE's settings page edits that file), which governs 'ek9 -t' and the IDE's test runs

Common causes

• Infinite loop, missing loop increment
• Waiting for network/file that never becomes available
• Deadlock in concurrent code

Examples

Every example is compiled by the EK9 compiler: the first fails exactly as captioned, the others fix it.

Fails with E82001: test execution timeout - reported by the test runner, not the compiler, when EK9_TEST_TIMEOUT_MS=1000 ek9 -t greeting.ek9 runs in this project.

bad/greeting.ek9

#!ek9
defines module error.e82001.bad

  defines function

    greeting()
      -> name as String
      <- result as String: `Hello, ${name}`

    pairsToGreet()
      -> guests as Integer
      <- pairs as Integer: 0
      waiting <- guests
      while waiting <> 0
        waiting -= 2
        pairs += 1

bad/dev/tests.ek9

#!ek9
defines module error.e82001.bad.tests

  references
    error.e82001.bad::greeting
    error.e82001.bad::pairsToGreet

  defines program

    @Test
    GreetingIsKept()
      assert greeting("EK9") == "Hello, EK9"

    @Test
    FourGuestsMakeTwoPairs()
      assert pairsToGreet(4) == 2

    @Test
    FiveGuestsMakeTwoPairs()
      assert pairsToGreet(5) == 2

$ EK9_TEST_TIMEOUT_MS=1000 ek9 -t greeting.ek9
ℹ️ Found 3 tests:
  3 assert (unit tests with assertions)

Executing 3 tests...

❌ FAIL error.e82001.bad.tests::FiveGuestsMakeTwoPairs [Assert] (nms)
    E82001: did not finish within the test time limit of 1000 ms (EK9_TEST_TIMEOUT_MS)
✅ PASS error.e82001.bad.tests::FourGuestsMakeTwoPairs [Assert] (nms)
✅ PASS error.e82001.bad.tests::GreetingIsKept [Assert] (nms)

Summary: 2 passed, 1 failed (3 total)
Types: 3 assert
Duration: nms
ek9: tests failed (exit 11)

Fixed: The loop ends for every number of guests

good/greeting.ek9

#!ek9
defines module error.e82001.good

  defines function

    greeting()
      -> name as String
      <- result as String: `Hello, ${name}`

    pairsToGreet()
      -> guests as Integer
      <- pairs as Integer: 0
      waiting <- guests
      while waiting > 1
        waiting -= 2
        pairs += 1

good/dev/tests.ek9

#!ek9
defines module error.e82001.good.tests

  references
    error.e82001.good::greeting
    error.e82001.good::pairsToGreet

  defines program

    @Test
    GreetingIsKept()
      assert greeting("EK9") == "Hello, EK9"

    @Test
    FourGuestsMakeTwoPairs()
      assert pairsToGreet(4) == 2

    @Test
    FiveGuestsMakeTwoPairs()
      assert pairsToGreet(5) == 2

$ EK9_TEST_TIMEOUT_MS=1000 ek9 -t greeting.ek9
ℹ️ Found 3 tests:
  3 assert (unit tests with assertions)

Executing 3 tests...

✅ PASS error.e82001.good.tests::FiveGuestsMakeTwoPairs [Assert] (nms)
✅ PASS error.e82001.good.tests::FourGuestsMakeTwoPairs [Assert] (nms)
✅ PASS error.e82001.good.tests::GreetingIsKept [Assert] (nms)

Summary: 3 passed, 0 failed (3 total)
Types: 3 assert
Duration: nms