EK9 Security: Input Sanitization and Sensitive Data

EK9 protects a program in two layers:

Both reject bad data where it enters (rejection at the source), and both work with no configuration. An organisation that needs its own policy or log format can extend them: InputSanitizer is open, and the log format is set in the environment.

Contents

The sanitized Parameter Modifier

Mark an incoming String parameter sanitized. The receiving function or method replaces the parameter with its own sanitized copy before any of its code runs. 

executeQuery()
  -> sql as sanitized String
  <- result as String?
  //The 'sql' parameter is sanitized on entry,
  //before any of this function's code runs - preventing SQL injection
  result: `executed ${sql}`

processUserInput()
  ->
    name as sanitized String
    comment as sanitized String
  <- success as Boolean?
  //Both parameters are sanitized before use: a threat leaves one unset
  success: name? and comment?

Full source: sanitized_modifier.ek9 - Download sanitized_modifier.ek9


How it works:

Every Route Is Covered

The cleaning is part of the receiver, so it runs however the receiver is reached: a direct call, a call through a super type or trait, an operator or a switch case, a stream stage (map with, filter by, ...), a sink or tee, a dispatcher, a callback from a built-in type such as Optional.whenPresent (which has no EK9 call site at all), and a program's command-line argument. The caller never writes sanitized (E07942). See ek9 -q 1399 for a runnable example.

Overriding: Add, Never Drop

sanitized is part of a method's or function's contract, like pure.

Extern Declarations

An extern declaration (defines extern module) cannot take a sanitized parameter (E07944): the compiler never generates an extern body, so it has nowhere to put the cleaning. Clean the input in the external implementation (in Java, with InputSanitizer), or wrap the extern call in an EK9 function whose parameter is sanitized. See ek9 -q 1400.

Important Restrictions

Target Sites vs Call Sites: The "Rejection at the Source" Philosophy

sanitized belongs on a function's or method's parameter definition, the target site where untrusted data first enters. It is an error anywhere else:

Why EK9 does it this way. Security reasoning is simpler when sanitization happens in one place:

  1. Entry points define where untrusted data enters.
  2. Everything inside the trust boundary is already safe.
  3. The compiler generates the cleaning at the start of every receiving function and method.

There is no doubt about who is responsible. The author of the function or method states the requirement; the compiler enforces it in the receiver itself, so no caller can bypass it.

Each wrong place is a compile error:

Fails with E07942: 'sanitized' is specified in the function/method definition, not at call sites

#!ek9
defines module bad.site.sanitized.callsite

  defines function

    processInput()
      -> userInput as sanitized String
      <- rtn as String: String(userInput)

    caller()
      -> userInput as String
      result <- processInput(sanitized userInput)  //The caller can't specify sanitization
      Stdout().println(result)

Download sanitized_callsite_bad.ek9

Fails with E07941: 'sanitized' cannot be used on captured variables - sanitize at the original entry point

#!ek9
defines module bad.site.sanitized.capture

  defines function

    BaseHandler() as abstract
      <- rtn as String?

    makeHandler()
      -> userInput as String
      <- rtn as BaseHandler?
      rtn: (sanitized userInput) is BaseHandler as function  //A capture is already inside the trust boundary
        rtn: String(userInput)

Download sanitized_capture_bad.ek9

Fails with E07943: 'sanitized' cannot be used on variable declarations - sanitize at the function/method entry point

#!ek9
defines module bad.site.sanitized.declaration

  defines function

    getValue()
      <- rtn as String: "Ada"

    caller()
      name <- sanitized getValue()  //Wrong place: sanitize the parameters of the function that receives input
      Stdout().println(name)

Download sanitized_declaration_bad.ek9

The right place is the definition, the entry point:

processInput()
  -> userInput as sanitized String  //OK: sanitized at the definition - the target site
  <- rtn as String: userInput

Full source: sanitized_modifier.ek9 - Download sanitized_modifier.ek9

The full source runs both functions, with input that holds no threat:

executed customer 42
true
Ada

Why Direct Assignment is Blocked

local <- sanitizedParam would make both variables refer to the same sanitized copy. A change through one would show through the other, a hidden alias that defeats the defensive copy. Copy it explicitly instead.

Fails with E07930: cannot assign from sanitized parameter - creates hidden copy semantics

#!ek9
defines module bad.site.sanitized.alias

  defines function

    processInput()
      -> userInput as sanitized String
      local <- userInput       //'local' would be an alias of 'userInput'
      local += " modified"     //...and this would modify 'userInput' too
      Stdout().println(local)

Download sanitized_alias_bad.ek9

#!ek9
defines module site.sanitized.alias

  defines function

    processInput()
      -> userInput as sanitized String
      local <- String(userInput)  //OK: an explicit copy
      local += " modified"        //Only affects 'local'
      Stdout().println(local)

  defines program
    CopyNotAlias()
      processInput("Ada")

Download sanitized_alias_good.ek9

Ada modified

InputSanitizer Class

InputSanitizer is EK9's threat detection engine, the same one the sanitized modifier uses. Call it yourself when you need to:

Constructors

//Default - always logs threats to stderr
defaultSanitizer <- InputSanitizer()
rtn: rtn and defaultSanitizer.isSafe(userInput)

//With an explicit reporter (here, stderr)
stderrSanitizer <- InputSanitizer(Stderr())
rtn: rtn and stderrSanitizer.isSafe(userInput)

Full source: sanitized_constructors.ek9 - Download sanitized_constructors.ek9


sanitize, sanitizeWithoutPathChecks and isSafe log every threat they find, and no method turns that off; detectThreat only names the threat. By default the log goes to stderr; pass a StringOutput to send it elsewhere (stdout, for example). A file's output is a resource that its try closes when the try ends, and a sanitizer keeps the reporter it is given, so the compiler rejects a file reporter given to the sanitizer (E81075).

The EK9_SANITIZER_LOG_FORMAT environment variable sets the format; its SILENT value suppresses the log, as does passing a StringOutput that discards what it is given.

Methods

#!ek9
defines module site.sanitized.methods

  defines function

    processData()
      -> text as String
      Stdout().println(text)

    handleThreatDetection()
      Stderr().println("Threat detected")

    logThreat()
      -> threatKinds as String
      Stderr().println(threatKinds)

  defines program
    SanitizerExample()
      sanitizer <- InputSanitizer(Stderr())

      //Sanitize input - returns unset if a threat is detected
      userInput <- "some user input"
      result <- sanitizer.sanitize(userInput)
      if result?
        //Safe to use
        processData(result)
      else
        //Threat detected, handle appropriately
        handleThreatDetection()

      //Check safety without modification
      if sanitizer.isSafe(userInput)
        processData(userInput)

      //Get the threat type(s) as a comma-separated string
      threat <- sanitizer.detectThreat(userInput)
      if threat?
        logThreat(threat)  //e.g. "SQL_INJECTION, XSS"

      //For environment variables (skip path traversal checks)
      envValue <- EnvVars().get("CONFIG_PATH")
      if envValue?
        cleanEnvValue <- sanitizer.sanitizeWithoutPathChecks(envValue)
        processData(cleanEnvValue)

Download sanitized_methods.ek9


Threat Types Detected

The categories follow the OWASP Top 10. The main ones:

Threat Type Description Example Pattern
SQL_INJECTION SQL injection patterns ' OR '1'='1, ; DROP TABLE
COMMAND_INJECTION Shell/OS command injection ; rm -rf /, | cat /etc/passwd
PATH_TRAVERSAL Directory traversal attacks ../../../etc/passwd
XSS Cross-Site Scripting <script>alert('xss')</script>
XXE XML External Entity attacks <!ENTITY xxe SYSTEM "file://">
SSTI Server-Side Template Injection {{7*7}}, ${T(java.lang.Runtime)}

It also detects PROMPT_INJECTION, DESERIALIZATION, SSRF, LDAP_INJECTION, NOSQL_INJECTION, CRLF_INJECTION, LOG_INJECTION and EMAIL_HEADER_INJECTION.

When one input holds several threats, the types come back comma-separated (for example "SQL_INJECTION, XSS"). A signature-based detector can match the same input too, so one SQL injection can log both SQL_INJECTION and SQL_INJECTION_SIGNATURE, as the runs below show.

Log Format Configuration

The EK9_SANITIZER_LOG_FORMAT environment variable sets the log format, so the log fits your existing SIEM with no code change. 

# Set log format (default: JSON)
export EK9_SANITIZER_LOG_FORMAT=JSON    # Simple JSON (universal, default)
export EK9_SANITIZER_LOG_FORMAT=ECS     # Elastic Common Schema
export EK9_SANITIZER_LOG_FORMAT=CEF     # Common Event Format
export EK9_SANITIZER_LOG_FORMAT=SIMPLE  # Simple bracket format [THREAT_TYPE]
export EK9_SANITIZER_LOG_FORMAT=SILENT  # Suppress all output (testing only)

JSON Format (Default)

One JSON object per line, readable by any log shipper or monitoring tool. The examples in this section run this program, which meets one SQL injection and one cross-site script: 

#!ek9
defines module site.sanitized.logformats

  defines program
    MeetThreats()
      //The default reporter is stderr; each threat is logged in the configured format
      sanitizer <- InputSanitizer()
      sqlInjection <- sanitizer.sanitize(String("' OR '1'='1"))
      crossSiteScript <- sanitizer.sanitize(String("<script>alert(1)</script>"))
      //Both are rejected: each result is unset
      Stdout().println(`Both rejected: ${not (sqlInjection? or crossSiteScript?)}`)

Download sanitized_logformats.ek9

$ ek9 sanitized_logformats.ek9
{"timestamp":"date-time","level":"warn","threat":"SQL_INJECTION","message":"Dangerous input detected: ' OR '1'='1"}
{"timestamp":"date-time","level":"warn","threat":"SQL_INJECTION_SIGNATURE","message":"Dangerous input detected: ' OR '1'='1"}
{"timestamp":"date-time","level":"warn","threat":"XSS","message":"Dangerous input detected: <script>alert(1)</script>"}
Both rejected: true

ECS Format (Elastic Common Schema)

ECS-aligned JSON for Elasticsearch, Splunk, Datadog, CloudWatch and Google Cloud Logging. With a SanitizationContext the event carries its source, service, field and trace (this is the context example below, run with EK9_SANITIZER_LOG_FORMAT=ECS): 

$ EK9_SANITIZER_LOG_FORMAT=ECS ek9 sanitized_context.ek9
{"@timestamp":"date-time","log.level":"warn","event.category":"intrusion_detection","event.type":"denied","event.action":"input_rejected","threat.indicator.type":"SQL_INJECTION","message":"Dangerous input detected","source.ip":"192.168.1.100","service.name":"UserService","trace.id":"trace-abc-123","ek9.field.name":"userId","ek9.field.source":"PATH","ek9.operation":"getUser","ek9.input.value":"' OR '1'='1","ecs.version":"8.11"}
{"@timestamp":"date-time","log.level":"warn","event.category":"intrusion_detection","event.type":"denied","event.action":"input_rejected","threat.indicator.type":"SQL_INJECTION_SIGNATURE","message":"Dangerous input detected","source.ip":"192.168.1.100","service.name":"UserService","trace.id":"trace-abc-123","ek9.field.name":"userId","ek9.field.source":"PATH","ek9.operation":"getUser","ek9.input.value":"' OR '1'='1","ecs.version":"8.11"}
{"@timestamp":"date-time","log.level":"warn","event.category":"intrusion_detection","event.type":"denied","event.action":"input_rejected","threat.indicator.type":"SQL_INJECTION","message":"Dangerous input detected","source.ip":"192.168.1.100","service.name":"UserService","ek9.operation":"getUser","ek9.input.value":"' OR '1'='1","ecs.version":"8.11"}
{"@timestamp":"date-time","log.level":"warn","event.category":"intrusion_detection","event.type":"denied","event.action":"input_rejected","threat.indicator.type":"SQL_INJECTION_SIGNATURE","message":"Dangerous input detected","source.ip":"192.168.1.100","service.name":"UserService","ek9.operation":"getUser","ek9.input.value":"' OR '1'='1","ecs.version":"8.11"}

CEF Format (Common Event Format)

CEF for ArcSight, Azure Sentinel, QRadar and LogRhythm: 

$ EK9_SANITIZER_LOG_FORMAT=CEF ek9 sanitized_context.ek9
CEF:0|EK9|InputSanitizer|1.0|SQL_INJECTION|Dangerous input detected|8|src=192.168.1.100 svc=UserService rt=n cs1=userId cs1Label=FieldName cs2=PATH cs2Label=FieldSource cs3=getUser cs3Label=Operation cs4=trace-abc-123 cs4Label=TraceId msg=' OR '1'\='1
CEF:0|EK9|InputSanitizer|1.0|SQL_INJECTION_SIGNATURE|Dangerous input detected|8|src=192.168.1.100 svc=UserService rt=n cs1=userId cs1Label=FieldName cs2=PATH cs2Label=FieldSource cs3=getUser cs3Label=Operation cs4=trace-abc-123 cs4Label=TraceId msg=' OR '1'\='1
CEF:0|EK9|InputSanitizer|1.0|SQL_INJECTION|Dangerous input detected|8|src=192.168.1.100 svc=UserService rt=n cs3=getUser cs3Label=Operation msg=' OR '1'\='1
CEF:0|EK9|InputSanitizer|1.0|SQL_INJECTION_SIGNATURE|Dangerous input detected|8|src=192.168.1.100 svc=UserService rt=n cs3=getUser cs3Label=Operation msg=' OR '1'\='1

CEF Severity Mapping:

SIMPLE Format

A minimal bracket format for scripts and debugging, when JSON is more than you need. No timestamp and no context fields, only the threat type and the input: 

$ EK9_SANITIZER_LOG_FORMAT=SIMPLE ek9 sanitized_logformats.ek9
[SQL_INJECTION] Dangerous input detected: ' OR '1'='1
[SQL_INJECTION_SIGNATURE] Dangerous input detected: ' OR '1'='1
[XSS] Dangerous input detected: <script>alert(1)</script>
Both rejected: true

SILENT Format

Suppresses all sanitizer output. The same program prints only its own line: 

$ EK9_SANITIZER_LOG_FORMAT=SILENT ek9 sanitized_logformats.ek9
Both rejected: true

Use it in tests where the sanitizer is not what you are testing, for example a test of code generation for a sanitized parameter, where the logs would clutter the output.

Warning: never use SILENT in production: you lose sight of attack attempts.

SanitizationContext

A record of metadata for security event logging. Pass one to an InputSanitizer method and the ECS and CEF formats include the context fields that are set.

Its API, as ek9 -h describes it:

$ ek9 -h SanitizationContext
EK9 Help for 'SanitizationContext'
========================================


--- TYPE API (org.ek9.lang) ---

    SanitizationContext as open
      SanitizationContext() as pure

      SanitizationContext() as pure
        ->
          timestamp as DateTime
          service as String
          operation as String
          fieldName as String
          fieldSource as String
          sourceIp as String
          traceId as String

      fieldName() as pure
        <- rtn as String?

      fieldSource() as pure
        <- rtn as String?

      operation() as pure
        <- rtn as String?

      service() as pure
        <- rtn as String?

      sourceIp() as pure
        <- rtn as String?

      timestamp() as pure
        <- rtn as DateTime?

      traceId() as pure
        <- rtn as String?

      operator #? as pure
        <- rtn as Integer?

      operator #^ as pure
        <- rtn as String?

      operator $ as pure
        <- rtn as String?

      operator :=:
        -> other as SanitizationContext

      operator :^:
        -> other as SanitizationContext

      operator :~:
        -> other as SanitizationContext

      operator <> as pure
        -> other as SanitizationContext
        <- rtn as Boolean?

      operator == as pure
        -> other as Any
        <- rtn as Boolean?

      operator == as pure
        -> other as SanitizationContext
        <- rtn as Boolean?

      override operator ? as pure
        <- rtn as Boolean?

Using SanitizationContext with InputSanitizer: 

#!ek9
defines module site.sanitized.context

  defines function

    processData()
      -> text as String
      Stdout().println(text)

  defines program
    ContextExample()
      sanitizer <- InputSanitizer()

      //Create a context with rich metadata
      context <- SanitizationContext(
        timestamp: SystemClock().dateTime(),
        service: "UserService",
        operation: "getUser",
        fieldName: "userId",
        fieldSource: "PATH",
        sourceIp: "192.168.1.100",
        traceId: "trace-abc-123"
      )

      userInput <- "' OR '1'='1"

      //Sanitize with context - the log carries the context fields
      result <- sanitizer.sanitize(userInput, context)
      if result?
        processData(result)

      //A partly set context is unset as a record, but the log still carries the fields that ARE set
      partial <- SanitizationContext(
        timestamp: SystemClock().dateTime(),
        service: "UserService",
        operation: "getUser",
        fieldName: String(),
        fieldSource: String(),
        sourceIp: "192.168.1.100",
        traceId: String()
      )
      partialResult <- sanitizer.sanitize(userInput, partial)
      if partialResult?
        processData(partialResult)

Download sanitized_context.ek9


The log includes every context field that is set and leaves out the rest. The ECS and CEF runs above are this program: it sanitizes once with a full context and once with a partly set one.

Like any record, a SanitizationContext is set only when every field is set, so the partly set context tests as unset with ?. The logger does not use that test; it writes what it has, so the second pair of events carries the source, service and operation but no field name, field source or trace. With no timestamp set, the event is stamped with the current time. Security teams get the metadata to investigate incidents, correlate attacks and spot patterns.

Integration Examples

TextFile Automatic Sanitization

When user input names a file to read, sanitize the path to prevent path traversal: 

#!ek9
defines module site.sanitized.textfile

  defines function
    readUserFile()
      -> filename as sanitized String
      <- content as String: String()

      //A path traversal attempt leaves 'filename' unset, so nothing is read
      if filename?
        content: cat TextFile(filename).input() | collect as String

Download sanitized_textfile.ek9


Web Service Input Handling

In a web service, put sanitized on path parameters, query parameters and request body fields: 

UserService for :/users open

  getUser() as GET for :/{userId}
    -> userId as sanitized String :=: PATH
    <- response as HTTPResponse: userResponse(userId)
    //userId is sanitized before any of this method's code runs

  searchUsers() as GET for :/search
    -> query as sanitized String :=: QUERY "query"
    <- response as HTTPResponse: userResponse(query)
    //and so is the query parameter

Full source: sanitized_service.ek9 - Download sanitized_service.ek9


Command-Line Argument Processing

When a command-line argument goes on to a file operation or an external command: 

#!ek9
defines module site.sanitized.cmdline

  defines function

    processFile()
      -> fileName as String
      Stdout().println(`Processing ${fileName}`)

  defines program
    ProcessFiles()
      -> argv as List of String

      sanitizer <- InputSanitizer(Stderr())

      for commandLineArgument in argv
        cleanArgument <- sanitizer.sanitize(commandLineArgument)
        if cleanArgument?
          processFile(cleanArgument)
        else
          Stderr().println("Rejected potentially dangerous argument")

Download sanitized_cmdline.ek9


Best Practices

Use the sanitized Modifier by Default

Give every function or method that accepts user-provided String input the sanitized modifier. The function then cleans its own input on every call, so no caller can forget to: 

//GOOD: default to sanitized for user input
processUserData()
  -> input as sanitized String
  <- rtn as String: String(input)

//Only omit sanitized for trusted internal data
processInternalData()
  -> trustedInput as String
  <- rtn as String: trustedInput

Full source: sanitized_practices.ek9 - Download sanitized_practices.ek9


Handle Unset Results Gracefully

A detected threat leaves the parameter unset. Design the business logic to handle that case with a suitable error message: 

createUser()
  -> username as sanitized String
  <- result as Boolean?

  if username?
    //Safe to process
    result: doCreateUser(username)
  else
    //Threat detected - apply the normal business validation message:
    //defence in depth without leaking what was detected
    result: false
    logValidationError("Invalid username format")

Full source: sanitized_practices.ek9 - Download sanitized_practices.ek9


Use InputSanitizer for Environment Variables

An environment variable may legitimately hold a path. Use sanitizeWithoutPathChecks() for it: 

sanitizer <- InputSanitizer(Stderr())
envVars <- EnvVars()

//Path traversal patterns are legitimate in environment variables
configPath <- envVars.get("CONFIG_PATH")
if configPath?
  rtn: sanitizer.sanitizeWithoutPathChecks(configPath)

Full source: sanitized_practices.ek9 - Download sanitized_practices.ek9


Logging to a SIEM

In production, set the log format to match your SIEM: 

#Production deployment - route stderr to SIEM
export EK9_SANITIZER_LOG_FORMAT=ECS   #For Elasticsearch/Splunk/Datadog
#or
export EK9_SANITIZER_LOG_FORMAT=CEF   #For ArcSight/Azure Sentinel/QRadar

Then route stderr to your log aggregator (Fluentd, Filebeat, ...) and the security events reach the SIEM.

Sensitive Type and Secret Detection

Input sanitization guards against malicious input at run time. The second layer guards against leaked credentials, at compile time and at run time.

Compile-Time Secret Detection

The compiler checks every string literal, including the text segments of interpolated strings, against known credential formats. A hard-coded secret is a compile error, so no program can be built with it and it never reaches a build artifact or production.

The categories:

Error Code Category Example Patterns
E11080 Cloud Provider AWS keys (AKIA...), GCP API keys (AIza...), Azure connection strings
E11081 Platform Token GitHub (ghp_), GitLab (glpat-), Slack (xoxb-), npm, Shopify, Heroku
E11082 Private Key PEM headers: RSA, EC, DSA, PKCS8, OPENSSH private keys
E11083 Database URL postgres://user:pass@host, MySQL, MongoDB, Redis, JDBC
E11084 JWT Token eyJhbGci... three-part header.payload.signature structure
E11086 API Key Stripe (sk_test_), Anthropic (sk-ant-), SendGrid (SG.), and more than 50 other API-key patterns

Together these are more than 100 patterns. Tools such as GitGuardian and TruffleHog look for the same kind of secret after it has been written; EK9 refuses to compile it.

The Sensitive Type

With secrets out of the source, they have to be loaded at run time. The Sensitive built-in type wraps a secret: 

#!ek9
defines module site.sanitized.sensitive

  defines function

    doHttpCall() as pure
      -> header as String
      <- response as String: `called with a ${length header} character header`

  defines class

    HttpClient with trait of Privileged
      apiKey as Sensitive?

      HttpClient()
        -> key as Sensitive
        apiKey: key

      default private HttpClient()

      sendRequest()
        <- response as String: String()
        //reveal() only works because HttpClient has the Privileged trait
        if apiKey?
          header <- apiKey.reveal()
          response: doHttpCall(header)

      default operator ?

  defines function

    demo()
      env <- EnvVars()
      //sensitiveGet() is the ONLY way to create a set Sensitive value
      key <- env.sensitiveGet("API_KEY")
      if key?
        client <- HttpClient(key)
        result <- client.sendRequest()

        stdout <- Stdout()
        //Safe: printing the key shows "***REDACTED***", not the secret
        stdout.println(`Key: ${key}`)
        if result?
          stdout.println(result)

Download sanitized_sensitive.ek9


Privileged is an auditable access boundary: search a codebase for with trait of Privileged and you have every class that can read a raw secret.

Two Layers Working Together

Input Sanitization Sensitive Type
Protects against Malicious input (SQL injection, XSS, etc.) Credential leakage (API keys, passwords, tokens)
When Runtime (at function entry points) Compile time (literals) + Runtime (redaction)
Mechanism sanitized modifier, InputSanitizer Sensitive type, Privileged trait
On failure Parameter becomes unset + threat logged Compilation fails (literals) or redacted output (runtime)

For the sanitized modifier:

For secret detection and the Sensitive type:

Next Steps