E82001: test execution timeout

Category: E81xxx-E83xxx Test configuration, execution and coverage · Classification: TEST_TIMEOUT · All error codes

What went wrong

The @Test program was still running when its time limit ran out, so it failed. Every @Test run (each case of a parameterised test too) has the same limit: EK9_TEST_TIMEOUT_MS milliseconds, 30000 (30 seconds) unless you set it. The test is stopped and the rest of the run goes on: a native test is killed with every process it started; the IDE's and an AI's test runs each have a JVM of their own, killed with everything it started; under 'ek9 -t' a JVM test's run is interrupted and abandoned, and 'ek9 -t' still ends when the other tests are done.

Worked examples

Compilable examples demonstrating this error and the correct fix:

  • ek9 -q 953 (How do @Test programs work with command-line arguments...)

Example error

In the [EXAMPLE] below, 'pairsToGreet' takes guests two at a time 'while waiting <> 0', which never ends for an odd number of guests, so the test 'FiveGuestsMakeTwoPairs' fails at the limit (run with EK9_TEST_TIMEOUT_MS=5000, so 5000 ms) and the other tests pass.

How to fix it

  1. Fix the loop or the blocking call that never ends
  2. Do not wait on a real network or file in a test: give it a stand-in that answers at once
  3. A test that is slow but correct: split it, or raise the limit - export EK9_TEST_TIMEOUT_MS=60000, or put EK9_TEST_TIMEOUT_MS=60000 in ~/.ek9/ek9.env (the IDE's settings page edits that file), which governs 'ek9 -t' and the IDE's test runs

Common causes

  • Infinite loop, missing loop increment
  • Waiting for network/file that never becomes available
  • Deadlock in concurrent code

Examples

Every example is compiled by the EK9 compiler: the first fails exactly as captioned, the others fix it.

Fails with E82001: test execution timeout - reported by the test runner, not the compiler, when EK9_TEST_TIMEOUT_MS=5000 ek9 -t greeting.ek9 runs in this project.

bad/greeting.ek9

#!ek9
defines module error.e82001.bad

  defines function

    greeting()
      -> name as String
      <- result as String: `Hello, ${name}`

    pairsToGreet()
      -> guests as Integer
      <- pairs as Integer: 0
      waiting <- guests
      while waiting <> 0
        waiting -= 2
        pairs += 1

bad/dev/tests.ek9

#!ek9
defines module error.e82001.bad.tests

  references
    error.e82001.bad::greeting
    error.e82001.bad::pairsToGreet

  defines program

    @Test
    GreetingIsKept()
      assert greeting("EK9") == "Hello, EK9"

    @Test
    FourGuestsMakeTwoPairs()
      assert pairsToGreet(4) == 2

    @Test
    FiveGuestsMakeTwoPairs()
      assert pairsToGreet(5) == 2
$ EK9_TEST_TIMEOUT_MS=5000 ek9 -t greeting.ek9
ℹ️ Found 3 tests:
  3 assert (unit tests with assertions)

Executing 3 tests...

❌ FAIL error.e82001.bad.tests::FiveGuestsMakeTwoPairs [Assert] (nms)
    E82001: did not finish within the test time limit of 5000 ms (EK9_TEST_TIMEOUT_MS)
✅ PASS error.e82001.bad.tests::FourGuestsMakeTwoPairs [Assert] (nms)
✅ PASS error.e82001.bad.tests::GreetingIsKept [Assert] (nms)

Summary: 2 passed, 1 failed (3 total)
Types: 3 assert
Duration: nms
ek9: tests failed (exit 11)

Fixed: The loop ends for every number of guests

good/greeting.ek9

#!ek9
defines module error.e82001.good

  defines function

    greeting()
      -> name as String
      <- result as String: `Hello, ${name}`

    pairsToGreet()
      -> guests as Integer
      <- pairs as Integer: 0
      waiting <- guests
      while waiting > 1
        waiting -= 2
        pairs += 1

good/dev/tests.ek9

#!ek9
defines module error.e82001.good.tests

  references
    error.e82001.good::greeting
    error.e82001.good::pairsToGreet

  defines program

    @Test
    GreetingIsKept()
      assert greeting("EK9") == "Hello, EK9"

    @Test
    FourGuestsMakeTwoPairs()
      assert pairsToGreet(4) == 2

    @Test
    FiveGuestsMakeTwoPairs()
      assert pairsToGreet(5) == 2
$ EK9_TEST_TIMEOUT_MS=5000 ek9 -t greeting.ek9
ℹ️ Found 3 tests:
  3 assert (unit tests with assertions)

Executing 3 tests...

✅ PASS error.e82001.good.tests::FiveGuestsMakeTwoPairs [Assert] (nms)
✅ PASS error.e82001.good.tests::FourGuestsMakeTwoPairs [Assert] (nms)
✅ PASS error.e82001.good.tests::GreetingIsKept [Assert] (nms)

Summary: 3 passed, 0 failed (3 total)
Types: 3 assert
Duration: nms

Found a problem?

If EK9 reported E82001 for a program you think is valid, or did not report it where it should have, use the compiler form with the smallest program that shows it. If this page is wrong or unclear, use the documentation form. Both are at github.com/ek9lang/ek9-issues.