How do I use a URL in EK9, and why can't I write one in my code?

← Security and Sanitization · Ref: Q2240

A URL your program calls comes from the ENVIRONMENT: 'Networking().urlFromEnv("ORDERS_URL")' reads the variable and parses it strictly into a URL. There is no URL constructor that takes text, because an endpoint is configuration - operations staff change it long after the code is written - and an address in the source is also an exposure that nobody reviewing the package can see. A missing variable or an invalid URL gives an UNSET URL, so the program checks it with '?' like any other value. Parsing is strict and never repairs: whitespace, backslashes, a 'user:password@' before the host and any scheme other than http/https are rejected, and the result is canonical - lower-case scheme and host, the default port removed, '.' and '..' segments resolved - so 'HTTPS://Orders.Example.COM:443/v1' and 'https://orders.example.com/v1' are the same URL. Other URLs are DERIVED from that base: 'withPath("/v2")', 'withPathSegment(segment)' (one segment, percent-encoded, so data such as 'item 7' or 'a/b' cannot reshape the path), 'withQueryParam(name, value)' and 'resolve("archive")' (a relative reference only). They keep the base's host, so data can never send the request somewhere else. A URL that ARRIVES while the program runs - a webhook a user registers, a link in another service's response - goes through 'Networking().urlOf(text)', which parses it the same way; the compiler refuses urlOf text it can know when it compiles (a literal, a constant, or a local only ever given those) with E08275, so urlOf cannot be used to hard-code an endpoint. A web service builds the links it sends back from its own public base address in the environment (for example PUBLIC_BASE_URL) and the withers, never from the request's Host header, which a client can forge. Operations can also limit which URLs the gateways accept: EK9_URL_POLICY names an allow/deny file (hosts, '*.' domains, URL prefixes, CIDR ranges such as 169.254.169.254/32), and a URL it refuses is unset and logged. Networking is a gated type: a package under capability enforcement declares both 'org.ek9.lang::Networking' and 'org.ek9.lang::EnvVars'.

Example

#!ek9
defines module qa.security.urls

  defines function

    //A webhook address a user registered: URL text that arrives while the program runs.
    webhookFrom()
      -> registered as String
      <- rtn as URL: Networking().urlOf(registered)

  defines program

    OrderLinks()
      stdout <- Stdout()
      net <- Networking()

      //The endpoint is configuration: operations set ORDERS_URL, the code never names it.
      orders <- net.urlFromEnv("ORDERS_URL")
      stdout.println(`orders: ${orders}`)
      stdout.println(`host: ${orders.host()}`)

      //Other URLs derive from that base, and keep its host.
      stdout.println(`item: ${orders.withPathSegment("item 7")}`)
      stdout.println(`page: ${orders.withQueryParam("page", "2")}`)

      //A missing or invalid setting is an unset URL - never a guess.
      archive <- net.urlFromEnv("ARCHIVE_URL")
      stdout.println(`archive set: ${archive?}`)

      //Text that arrives at run time goes through urlOf and is parsed the same way.
      env <- EnvVars()
      webhook <- webhookFrom(env.get("REGISTERED_WEBHOOK"))
      stdout.println(`webhook: ${webhook}`)
      rejected <- webhookFrom(env.get("REJECTED_WEBHOOK"))
      stdout.println(`rejected set: ${rejected?}`)

Common mistakes

E08275 — The text given to urlOf is a literal, so the endpoint would be hard-coded into the program. Put it in the environment and read it with urlFromEnv; keep urlOf for URL text that arrives at run time. See ek9 -h E08275 for details.

Incorrect:

      orders <- net.urlOf("https://orders.example.com/v1")

Correct:

      orders <- net.urlFromEnv("ORDERS_URL")

Coming from another language?

Java: 'new URI("https://...")' or a string constant is the usual pattern, so endpoints end up compiled in and changing one means a code change; 'URLEncoder.encode' is HTML form encoding (a space becomes '+'), not path encoding. Python: a module-level BASE_URL string and 'urllib.parse.urljoin', which happily joins an absolute URL from data and changes the host. Go: 'url.Parse' accepts a literal anywhere and 'ResolveReference' also follows an absolute reference. In EK9 the endpoint is an environment setting, derivation cannot change the host, and text known at compile time cannot become a URL.

Keywords: E08275, Networking, URL, base URL, endpoint, environment variable, urlFromEnv, urlOf, webhook, withPath, withPathSegment, withQueryParam