How do I write black-box tests with expected output files in EK9?

← Testing · Ref: Q204

EK9 black-box tests validate program behavior by comparing stdout output against an expected_output.txt companion file. No assertions needed.

BLACK-BOX TEST PATTERN

Mark a program with @Test:

  @Test
  MyTestProgram()
    stdout <- Stdout()
    stdout.println("Expected output")

EXPECTED OUTPUT FILE

Create a companion expected_output.txt in the same directory:

  Expected output

The test runner compares stdout against this file line by line.

NO ASSERTIONS NEEDED

The output IS the test. Print values and let the file comparison verify correctness. This eliminates testing framework boilerplate.

WHEN TO USE BLACK-BOX TESTS

Ideal for: integration tests, output formatting, end-to-end workflows, and any test where verifying printed output is the goal.

See Q205 for parameterized tests. See Q155 for basic unit testing. See Q156 for assertions. See Q208 for dynamic output placeholders.

Example

defines module qa.testdeep.blackbox

  defines program

    // === BLACK-BOX TEST ===
    // A @Test program prints to stdout.
    // A companion expected_output.txt file
    // contains the expected output.
    // The test runner compares them automatically.

    // Note: This file demonstrates the pattern.
    // In a real test directory, you would have:
    //   myTest/myTest.ek9
    //   myTest/expected_output.txt

    BlackBoxTestingDemo()
      stdout <- Stdout()

      stdout.println("Black-box testing pattern:")
      stdout.println("  1. Write a @Test program")
      stdout.println("  2. Print expected values to stdout")
      stdout.println("  3. Create expected_output.txt file")
      stdout.println("  4. Test runner compares output")
      stdout.println("No assertions needed")

Common mistakes

E50010 — Variables must be declared before use. Using stdout before its declaration is a forward reference. See ek9 -h E50010 for details.

Incorrect:

stdout.println("Black-box testing pattern:")
      stdout <- Stdout()

Correct:

stdout <- Stdout()

      stdout.println("Black-box testing pattern:")

Coming from another language?

Java: JUnit requires assertion methods and output stream capture. Python: pytest capsys fixture for output capture. Rust: custom output capture helpers. Go: testing.T with stdout redirection. EK9: @Test program with expected_output.txt companion file, no assertions or framework API needed.

Keywords: output, coverage, test, verify, file, stdout, black-box, comparison, expected