How do @Test programs work with command-line arguments, timeouts, and expected output?

← Testing · Ref: Q953

@Test programs accept TYPED PARAMETERS (not just String) — the EK9 runtime introspects the program's parameter signature and parses commandline_arg lines into the declared types. The E81xxx codes flag config issues (paired-file naming, arg counts, type conversion); the E82xxx codes flag execution issues (timeout, no-output).

TEST PROGRAM SIGNATURES

Tests ARE programs. They CAN have typed parameters:

  @Test
  NumberProcessor()
    ->
      num1 as Integer
      num2 as Integer
    assert num1?
    assert num2?

The runtime auto-parses each commandline_arg line into the declared type — Integer, Date, Duration, DateTime, Money, Boolean, etc. Per parameterizedHybrid.ek9 corpus example.

COMPANION FILES (live in test directory)

  expected_output.txt — single-case stdout (parameter-less test)
  expected_output_t1.txt / _t2.txt — numbered multi-case stdout (at PACKAGE ROOT)
  commandline_arg_<name>.txt + expected_case_<name>.txt — named-case pair (INSIDE dev/)
  env_vars.txt — KEY=VALUE environment
  expected_stderr.txt — expected stderr (supports {{DateTime}} / {{Millisecond}})
  test_data.txt — auxiliary input data

E81xxx CONFIG ERRORS (per TestConfigurationError.java)

  E81001 MISSING_EXPECTED_OUTPUT — commandline_arg_X.txt without expected_case_X.txt
  E81002 ORPHAN_EXPECTED_OUTPUT — expected_case_X.txt without commandline_arg_X.txt
  E81003 ARGUMENT_COUNT_MISMATCH — program declares N params, file provides not-N args
  E81004 UNEXPECTED_ARGUMENTS — commandline file exists but program has no params
  E81005 MISSING_ARGUMENTS — program declares params but no commandline files found
  E81006 TYPE_CONVERSION_ERROR — string can't parse as declared parameter type
  E81007 EMPTY_TEST — @Test program has no asserts AND no expected_output
  E81008 DUPLICATE_CASE_ID — same case name used twice in one directory
  E81009 INVALID_FILE_PATTERN — typo in filename (fuzzy-match suggestion)
  E81010 INVALID_PLACEHOLDER — bad {{...}} syntax in expected output
  E81013 ENVVARS_PARSE_ERROR — malformed line in env_vars.txt (no =)
  E81014 ORPHAN_ENVVARS — env_vars file without matching commandline_arg
  E81016 ORPHAN_STDIN — orphan stdin file

E82xxx EXECUTION ERRORS

  E82001 TEST_TIMEOUT — execution exceeded timeout (default 30s, EK9_TEST_TIMEOUT_MS configurable)
  E82002 NO_OUTPUT — test ran but produced no stdout despite expected_output.txt

STDOUT MISMATCH

Note: stdout-content-doesn't-match-expected is NOT a numbered E81xx error. It's reported in the test runner's pass/fail report.

See Q1296 for capture isolation. See Q204 for black-box patterns. See Q205 for parameterized tests.

Example

defines module qa.testing.programargs

  defines program

    <?-
      Demonstrates the @Test program pattern with typed parameters
      and the companion-file model. In a real test, prefix with @Test.
      Programs (and tests) accept typed parameters that the EK9 runtime
      auto-parses from commandline_arg files.
    -?>
    TestProgramArgsDemo()
      stdout <- Stdout()

      stdout.println("@Test program facts:")
      stdout.println("  Tests ARE programs (defines program + @Test directive)")
      stdout.println("  Typed parameters supported (Integer, Date, Money, etc.)")
      stdout.println("  Runtime auto-parses commandline_arg lines into types")
      stdout.println("  E81xxx codes flag config issues (paired files, arg counts, type conversion)")
      stdout.println("  E82001 = TEST_TIMEOUT (default 30s)")
      stdout.println("  E82002 = NO_OUTPUT (test ran but produced no stdout)")
Other ways to ask this
  • What triggers E81001 MISSING_EXPECTED_OUTPUT?
  • What triggers E81002 ORPHAN_EXPECTED_OUTPUT?
  • What triggers E81003 ARGUMENT_COUNT_MISMATCH?
  • What triggers E81004 UNEXPECTED_ARGUMENTS?
  • What triggers E81005 MISSING_ARGUMENTS?
  • What triggers E81006 TYPE_CONVERSION_ERROR?
  • What triggers E81007 EMPTY_TEST?
  • What triggers E81008 DUPLICATE_CASE_ID?
  • What triggers E82001 TEST_TIMEOUT?
  • What triggers E82002 NO_OUTPUT?
  • Can @Test programs accept typed parameters?
  • How do I pass arguments to test programs?

Coming from another language?

Java JUnit: @Test annotation, @ParameterizedTest with @ValueSource for typed params. Python pytest: fixtures + parametrize for typed params. Go: testing.T with table-driven tests. Rust: #[test] with proptest for parameterised. EK9: @Test directive on a program that accepts typed parameters; the runtime auto-parses commandline_arg files into the declared types, no manual parsing or annotations needed.

Keywords: E81002, E81001, test, E82002, E82001, typed parameters, expected_case, program, output, expected, commandline_arg, E81003, E81005, E81007, E81008, timeout, E81006, E81004, env_vars, argument