EK9 Network Types

These types are for network communications and protocols.

The Types

Worked Example

Each network type above has examples, with some explanation. They assume you have read Built in Types, Collection Types and Standard Types; read those first if you have not.

UDP Examples

There are two server and two client examples for UDP, from imperative to more functional examples with Stream pipelines.

A UDP connection receives packets of up to 1024 bytes unless you set packetSize in its NetworkProperties; larger packets may be more likely to be dropped by the network. UDP packets are not guaranteed to be delivered. That is the nature of UDP, not of EK9, and it suits applications that can accept some data loss.

UDP Server (1)

An imperative server that echoes UDP messages until it receives 'end'. It listens on port 4445, blocks to receive() each packet and echoes it back. 

#!ek9
defines module site.networkTypes.udpServer
  defines program
    UDPServer1()
      stdout <- Stdout()
      try
        -> udpConnection <- UDP(NetworkProperties(port: 4445))

        //The loop stops on a condition; the try closes the connection when it ends.
        serving <- true
        while serving and udpConnection?
          packet <- udpConnection.receive()
          if packet?
            if packet.content == "end"
              endPacket <- UDPPacket(packet.properties, "OK will stop Server")
              udpConnection.send(endPacket)
              serving: false
            else
              udpConnection.send(packet)

      stdout.println("UDP Server Complete")

Download networkTypes_udp_server.ek9


With the iterator syntax, the receiving part above: 

while serving and udpConnection?
  packet <- udpConnection.receive()

Full source: networkTypes_udp_server.ek9 - Download networkTypes_udp_server.ek9

while serving and udpConnection.hasNext()
  packet <- udpConnection.next()

Full source: networkTypes_udp_server_iterator.ek9 - Download networkTypes_udp_server_iterator.ek9

This is the imperative style familiar from Java, C, C++, C# or Python. The next example takes a more functional approach.

UDP Server (2)

This does broadly the same as Server (1), except that it:

A Stream pipeline continuously accepts incoming UDP packets, passes them through a function (which echoes the content back) and sends what it echoed to 'Standard Out'. 

#!ek9
defines module site.networkTypes.udpServerSignals
  defines function
    UDPPacketHandler() as abstract
      -> incoming as UDPPacket
      <- outgoing as UDPPacket?

  defines program
    UDPServer2()
      ->
        port as Integer
        packetSize as Integer
        signal as String

      stdout <- Stdout()
      try
        -> udpConnection <- UDP(NetworkProperties(port, packetSize))

        //This time stop server with abort signal not content 'end'.
        //A handler that returns a set Integer ends the process with that exit code.
        //It does not close the connection it could capture - only the try that owns it does.
        abortHandler <- () is SignalHandler as function
          result: 0
          stderr <- Stderr()
          stderr.println("Received abort Signal stopping server")

        Signals().register(signal, abortHandler)

        echoFunction <- (udpConnection) is UDPPacketHandler
          outgoing = incoming
          udpConnection.send(outgoing)

        cat udpConnection | map by echoFunction > stdout
      stdout.println("UDP Server Complete")

Download networkTypes_udp_server_signals.ek9


There is more code because it does more: it handles command line parameters (these could come from Environment Variables instead) and it ends on a signal rather than on message content.

It processes each incoming packet with a dynamic function that extends an abstract function, so you can add any processing before and after the echo. For example, you could:

Why EK9 does it this way. The functional approach gives more flexibility and room for future extension than mixing all that logic into the imperative processing loop. (That is an aside to what the UDP class does.)

UDP Client (1)

A client that connects to one of the servers above. 

#!ek9
defines module site.networkTypes.udpClient
  defines program
    UDPClient1()
      stdout <- Stdout()
      stderr <- Stderr()
      localhost <- Networking().loopbackV4()

      try //use a try block with a closable resource, auto close
        -> client <- UDP(NetworkProperties(timeout: 250ms))

        packet1 <- UDPPacket(NetworkProperties(localhost, 4445), "A Test")
        stdout.println(`Sending [${packet1}]`)
        client.send(packet1)
        stdout.println("Waiting for response")
        response1 <- client.receive()
        if response1?
          stdout.println(`Received [${response1}]`)
        else
          stderr.println(`There was no response [${client.lastErrorMessage()}]`)

        packet2 <- UDPPacket(NetworkProperties(host: localhost, port: 4445), "end")
        stdout.println(`Sending [${packet2}]`)
        client.send(packet2)
        stdout.println("Waiting for response")
        response2 <- client.receive()
        if response2?
          stdout.println(`Received [${response2}]`)
        else
          stderr.println(`There was no response [${client.lastErrorMessage()}]`)

      stdout.println("UDP Client Complete")

Download networkTypes_udp_client.ek9


It also uses the UDP class, but with no port. The UDP instance is created, with a timeout for receiving packets, as the incoming parameter of the try, so it is closed automatically.

Where to send the message is bound into the UDPPacket when it is created; the packet is then sent through the UDP class.

This client expects a response, which depending on the server logic may not always come. It allows for unreliable transmission with a timeout of 250 milliseconds, and checks that the response packet is valid: when no packet arrives within the timeout, receive() returns an unset UDPPacket.

This client is imperative/procedural in style.

UDP Client (2)

A more functional client for the same servers, which also processes command line parameters. 

#!ek9
defines module site.networkTypes.udpClientStream
  defines type
    List of String

  defines function
    packetConverter() as abstract
      -> content as String
      <- outgoing as UDPPacket?

  defines program
    UDPClient2()
      ->
        port as Integer
        timeout as Millisecond
      host <- Networking().loopbackV4()
      try
        -> udpConnection <- UDP(NetworkProperties(timeout))

        //This is the nearest we get to a lambda, inline dynamic function
        //You must use () for the single line you are allowed.
        cat ["A Test", "Middle Item", "end" ]
          | map (host, port) extends packetConverter (outgoing: UDPPacket(NetworkProperties(host, port), content))
          > udpConnection

      Stdout().println("UDP Client Complete")

Download networkTypes_udp_client_stream.ek9


Like the second server, it takes command line parameters, and sets up the UDP connection as the first client does. This time the messages are held in a List of String and pipelined through a dynamic function that wraps each one in a UDPPacket with the host address and port. They are then pushed out through the UDP connection with the > operator.

The dynamic function is simple and inlined here. It could have been held as a function delegate or created by a higher order function.

This client does not check whether the server responded. The UDP connection was opened as the incoming parameter of the try, so it is closed automatically.

Why EK9 does it this way. There is something about the functional approach that is more elegant, in my opinion, though I'm less keen on the inline dynamic function.

UDP Summary

You can work with UDP procedurally or functionally. UDP is useful for a range of applications, as long as you accept its limitations.

TCP Examples

There are several client and server examples for TCP, each using TCP differently. TCP is connection oriented and more reliable than UDP.

TCP Server (1)

This TCP server is procedural. The key difference from UDP is that TCP is connection oriented, so a server must service multiple client connections concurrently.

The TCP constructor takes a NetworkProperties record, with properties that control: 

#!ek9
defines module site.networkTypes.tcpServer
  defines program
    TCPServer1()
      stdout <- Stdout()
      stderr <- Stderr()

      stdout.println(`Version of program package is ${Package().version}`)

      try
        -> tcpServer <- TCP(NetworkProperties(port: 4446, backlog: 5, maxConcurrent: 4, localOnly: true))

        if tcpServer?
          handler <- (tcpServer, stderr) extends TCPHandler as function
            stderr.println("Connection made")
            while input?
              line <- input.next()
              stderr.println(`Received [${line}]`)
              output.println(line.trim().upperCase())
              if line == "shutdown"
                tcpServer.stop()
            stderr.println("No more input on connection")

          stdout.println("Awaiting connections")
          if tcpServer.accept(handler)
            stdout.println("TCP Server completed and shutdown")
          else
            stderr.println(`Was not able to service requests gracefully [${tcpServer}]`)
        else
          stderr.println(`Failed to get TCP server running [${tcpServer}]`)

Download networkTypes_tcp_server.ek9


TCPHandler is a built-in abstract function; ek9 -h TCPHandler shows its signature: 

$ ek9 -h TCPHandler
EK9 Help for 'TCPHandler'
========================================


--- TYPE API (org.ek9.lang) ---

    TCPHandler as abstract
      ->
        input as StringInput
        output as StringOutput

So in your handler, the input and output arguments are inferred.

The TCPHandler processes the requests TCP clients make. Multiple clients call it at the same time, so it should not hold state unless that state is protected and designed for simultaneous access (more on this later).

tcpServer.accept(handler) blocks and waits for clients to connect (on port 4446 here). When a client connects, handler(input, output) is called and services the client's requests through the input and output, the connection from and to the client, until the client disconnects.

Meanwhile the tcpServer goes back to listening for more connections while the handler is still running. That is where multi-threading comes in.

If the client sends "shutdown", the handler calls tcpServer.stop(): the server stops listening, accept returns and the program completes and exits. You would not normally code it like this; the next example shows an alternative.

The handler stops the server; it does not close it. The server is a resource of the try, and only its owner closes a resource. Every close is generated, so the handler cannot close it at all:

stop() ends serving without closing: the server stays set until the end of the try closes it. The same applies to HTTP: serve() returns once stop() is called, for example from a signal handler.

TCP Server (2)
#!ek9
defines module site.networkTypes.tcpTwoServers

  defines function
    serverFunction() as abstract
      <- response as String?

    serviceConnections()
      ->
        tcpServer as TCP
        handler as TCPHandler
      <-
        response as String: `TCP Server on ${tcpServer} completed and shutdown`

      if not tcpServer.accept(handler)
        response: `Was not able to service requests gracefully on ${tcpServer}`

    createShutdownHandler
      ->
        controlServer as TCP
        processingServer as TCP
        shutdownCommand as String
      <-
        rtn as TCPHandler: (controlServer, processingServer, shutdownCommand) is TCPHandler as function
          while input?
            command <- input.next()
            if command == shutdownCommand
              output.println("Shutting down")
              processingServer.stop()
              controlServer.stop()
            else
              output.println(`Ignoring [${command}]`)

    createHandler()
      <- rtn as TCPHandler: () is TCPHandler as function
          cat input | map toUpperCase > output

    toUpperCase()
      -> input as String
      <- output as String: input.upperCase()

    shutdownFunction()
      -> tcpClient as TCP
      <- response as String: `Connection failed ${tcpClient}`

      if tcpClient?
        try
          -> connection <- tcpClient.connect()
          if connection?
            connection.output().println("SHUTDOWN")
            response: `Received response [${connection.input().next()} from ${tcpClient}]`
          else
            response: `Failed to connect on [${tcpClient}]`
      else
        response: `Failed to get TCP client running [${tcpClient}]`

  defines program
    TCPServer2()
      ->
        processingPort as Integer
        controlPort as Integer
        shutdownCommand as String

      stdout <- Stdout()

      //Alternative layout as there are a number of properties being set.
      //Both servers are named, so both are try-with-resources resources, closed when the try ends.
      try
        ->
          processingServer <- TCP(
            NetworkProperties(
              port: processingPort,
              timeout: 250ms,
              backlog: 2,
              maxConcurrent: 4,
              localOnly: false)
            )
          controlServer <- TCP(NetworkProperties(port: controlPort, localOnly: true))

        if controlServer? and processingServer?

          shutdownControl <- (controlServer, processingServer, shutdownCommand) extends serverFunction
            response: serviceConnections(
              tcpServer: controlServer,
              handler: createShutdownHandler(controlServer, processingServer, shutdownCommand)
            )

          processing <- (processingServer) is serverFunction
            response: serviceConnections(
              tcpServer: processingServer,
              handler: createHandler()
            )

          //Now run both of those serverFunctions concurrently.
          cat [shutdownControl, processing] | async > stdout
          //Once the serverFunctions both finish this program will finally exit.
        else
          Stderr().println(`Failed to start servers, control: ${controlServer} processing: ${processingServer}`)

Download networkTypes_tcp_two_servers.ek9


This server is much longer, but does more of what a real application does. The program TCPServer2 again processes command line parameters, then creates two TCP servers:

Next it creates two dynamic functions of type serverFunction, each returning the result of serviceConnections, which accepts a TCPHandler to deal with connected clients.

Finally a pipeline streams the two functions from a list through the async command, which runs them concurrently. Here that is critical.

The program then blocks in tcpServer.accept(handler) for both servers, and as clients connect the appropriate handler deals with each one.

Why EK9 does it this way. Network communications (and signal handling) are inherently multi-process/threaded/concurrent. EK9 lets you set the stage with handlers and functions, then runs them concurrently through the async part of a pipeline, which deals with joining the calls. There is no call back hell: 'multi-tasking' is implied but not explicit, so how EK9 manages the two TCP services is an implementation detail, not the developer's concern.

TCP Client (1)

A procedural TCP client. The connection to the server is closed at the end of the try; the server detects that there is no more input and closes its connections to this client. 

#!ek9
defines module site.networkTypes.tcpClient
  defines program
    TCPClient1()
      stdout <- Stdout()
      stderr <- Stderr()
      localhost <- Networking().loopbackV4()

      try
        -> tcpClient <- TCP(NetworkProperties(localhost, 4446))

        if tcpClient?
          stdout.println(`Will attempt to connect to server ${tcpClient}`)
          try
            -> connection <- tcpClient.connect()
            if connection?
              connection.output().println("Short Message")
              stdout.println(`Received response [${connection.input().next()}]`)
              //End of the try will just close the connection.
            else
              stderr.println(`Failed to connect [${tcpClient}]`)
        else
          stderr.println(`Failed to get TCP client running [${tcpClient}]`)

Download networkTypes_tcp_client.ek9


It is short and to the point; the next client does a little more.

TCP Client (2)

A more functional client for TCP Server (2): it connects to both the processing port and the control port, to shut the server application down. 

#!ek9
defines module site.networkTypes.tcpClientStream

  defines function
    shutdownFunction()
      -> tcpClient as TCP
      <- response as String: `Connection failed ${tcpClient}`

      if tcpClient?
        try
          -> connection <- tcpClient.connect()
          if connection?
            connection.output().println("SHUTDOWN")
            response: `Received response [${connection.input().next()} from ${tcpClient}]`
          else
            response: `Failed to connect on [${tcpClient}]`
      else
        response: `Failed to get TCP client running [${tcpClient}]`

    processingFunction() as abstract
      -> input as String
      <- response as String?

  defines program
    TCPClient2()
      ->
        processingPort as Integer
        controlPort as Integer

      stdout <- Stdout()
      localhost <- Networking().loopbackV4()
      try
        -> processingClient <- TCP(NetworkProperties(localhost, processingPort))

        if processingClient?
          try
            -> connection <- processingClient.connect()
            sender <- (connection) is processingFunction as function
              connection.output().println(input)
              response: connection.input().next()

            cat ["one", "two", "Alpha", "Beta", "Zeta", "end", "begin"] | map with sender > stdout

      //The control client is closeable, so it is acquired in a try and lent to the function.
      try
        -> controlClient <- TCP(NetworkProperties(localhost, controlPort))
        stdout.println(shutdownFunction(controlClient))

Download networkTypes_tcp_client_stream.ek9


Two ideas are worth a look:

A TCP Server and Its Own Client

A program that runs a server and a client against it (a test, typically) must not let the client race the server. acceptUntilComplete(handler, readyHandler) serves connections with handler and runs readyHandler on its own thread once the server is listening, passing it the serving TCP.

There are two ready function types. A TCPReadyHandler can act: connect, write, stop the server. A TCPReadyObserver is pure: it can only observe that the server is listening. Both take -> server as TCP. Port 0 lets the operating system choose a free port, which TCP writes back into the NetworkProperties the ready handler captured. 

#!ek9
defines module site.networkTypes.tcpReady

  defines function

    echo() is TCPHandler
      ->
        input as StringInput
        output as StringOutput
      while input?
        output.println(`echo:${input.next()}`)

  defines program

    TCPServerAndItsClient()
      stdout <- Stdout()
      props <- NetworkProperties(port: 0, backlog: 2, maxConcurrent: 2, localOnly: true)

      try
        -> tcpServer <- TCP(props)
        client <- (stdout, props) is TCPReadyHandler as function
          loopback <- Networking().loopbackV4()
          try
            -> connection <- TCP(NetworkProperties(loopback, props.port)).connect()
            if connection?
              connection.output().println("hello")
              stdout.println(`Received [${connection.input().next()}]`)
          server.stop()

        if tcpServer.acceptUntilComplete(echo, client)
          stdout.println(`Served until the client completed, server still set ${tcpServer?}`)
      stdout.println("The try has closed the server")

Download networkTypes_tcp_ready.ek9

Run, it prints: 

Received [echo:hello]
Served until the client completed, server still set true
The try has closed the server

HTTP has the same hook, serveUntilComplete(readyHandler) with an HTTPReadyHandler or a pure HTTPReadyObserver: the way to test a web service in one program (see ek9 -q 1760).

TCP Summary

You can build TCP clients and servers procedurally or functionally, with a reasonable level of control over the connection and the possibility of handling multiple concurrent connections.

HTTP

EK9 Web Services are built on the HTTP server.

Why EK9 does it this way. The built-in HTTP server is not designed to be as configurable as Tomcat, Jetty, NGINX or IIS. The expectation is that you front any EK9 web implementation with a caching proxy server, and deploy instances of your application in a range of scalable containers.

EK9 Web Services can serve simple html content, but they are designed to serve dynamic content, and have built-in logic and types to support full REST services.

The built-in types HTTPRequest and HTTPResponse have the structures to support the REST maturity model up to level 2. For level 3 (HATEOAS) you define and provide the content (HAL/ATOM or bespoke). etag and last modified are 'baked in': when your response gives either, the server sends the header and answers a conditional request (If-None-Match or If-Modified-Since) with 304 Not Modified. Checking that a PUT, PATCH or DELETE refers to the up-to-date version is up to your service. RESTFULAPI has more detail if you are new to REST.

serve() blocks until stop() is called, typically from a signal handler or an admin operation. To run a service and a client against it in one program (a test), use serveUntilComplete(readyHandler): it binds the application's services, starts listening and only then runs the ready handler on its own thread, passing it the serving HTTP. An HTTPReadyHandler can act (make requests and print the responses), and serving ends when it calls server.stop() or ends (normally, by Exception or by Panic); a pure HTTPReadyObserver can only observe that the server is up. Either way the server is stopped and drained, never closed (the try that made it closes it), and an Exception or Panic is rethrown to the caller. ek9 -q 1760 shows a complete program; the TCP twin is acceptUntilComplete.

Network Utility Types

Besides the TCP/UDP/HTTP socket types, EK9 has network utility types, used throughout the web services and networking stack. They are secure by default and handle network addresses in a type-safe way.

IPAddress

IPv4 and IPv6 address type. Uses InetAddress.ofLiteral() internally — never triggers DNS resolution, making it safe for untrusted input. Zone IDs are rejected. Invalid input results in an unset instance.

Predicates: isV4(), isV6(), isLoopback(), isPrivate(), isMulticast(), isLinkLocal(), isAnyLocal(), version().

CIDR

Network range (address/prefix notation, e.g. 10.0.0.0/8 or 2001:db8::/32). Host bits are silently normalized. Methods: networkAddress(), prefixLength(), includesNetwork(CIDR), isV4(), isV6(). The contains operator checks if an IPAddress falls within the CIDR range. Cross-version containment (IPv4 in IPv6 CIDR) always returns false.

Hostname

RFC 1123 validated hostname. Input is lowercased, trailing dots stripped, and internationalised domain names converted to ASCII via IDN.toASCII(). All-numeric TLDs are rejected. Methods: labels() returns List of String, resolve() returns IPAddress via DNS lookup.

URL

An absolute http or https URL, parsed once, strictly, into typed parts. An endpoint your program calls comes from the environment, through Networking().urlFromEnv("ORDERS_URL"): there is no way to make a URL from text in your source. An endpoint is configuration - operations staff change it long after the code is written - and keeping it out of the code also makes the exposure visible. A URL that arrives while the program runs - a webhook a user registers, a link in another service's response - goes through Networking().urlOf(text), which parses it the same way; the compiler refuses urlOf text it can know when it compiles - a literal, a constant, or a local only ever given those - with E08275. New URLs are derived from one of these: withPath("/v2/items"), withPathSegment(segment) (one segment taken from data, percent-encoded so it cannot reshape the path), withQueryParam(queryName, queryValue) and resolve("items/7") (a relative reference only; an absolute one is refused).

Links your service sends - a Location header, links in a JSON response, the next page - start from the service's public base address in the environment (for example PUBLIC_BASE_URL, set per deployment) and are built with the withers: base.resolve("orders").withPathSegment(orderId), sent as text with $. Do not build them from the incoming request's Host header: a client can forge it and make your service hand out links to another site. A relative link such as "/orders/42" is plain text and needs no URL at all.

Here a service builds the links it sends back. The base comes from PUBLIC_BASE_URL, the order id is data, and resolve("orders") keeps the base's /api/ path: 

#!ek9
defines module site.networkTypes.urlLinks

  defines program

    OrderLinks()
      -> orderId as String
      stdout <- Stdout()

      //The service's public address is set per deployment; the code never names it.
      base <- Networking().urlFromEnv("PUBLIC_BASE_URL")
      if base?
        //The order id is data: encoded as ONE path segment, so a '/' in it cannot reshape the path.
        order <- base.resolve("orders").withPathSegment(orderId)
        stdout.println(`order: ${order}`)
        stdout.println(`segments: ${order.pathSegments()}`)
        stdout.println(`next page: ${base.resolve("orders").withQueryParam("page", "2")}`)
      else
        stdout.println("PUBLIC_BASE_URL is not set, or is not an http or https URL")

Download networkTypes_url_links.ek9

Run with the variable set (to a spelling operations might type, upper case and default port included), the links are canonical, and the / in the order id is encoded so it stays one segment: 

$ PUBLIC_BASE_URL=HTTPS://Shop.Example.COM:443/api/ ek9 networkTypes_url_links.ek9 A/42
order: https://shop.example.com/api/orders/A%2F42
segments: ["api", "orders", "A/42"]
next page: https://shop.example.com/api/orders?page=2

Without it, the URL is unset and the program says so - nothing is guessed: 

$ ek9 networkTypes_url_links.ek9 A/42
PUBLIC_BASE_URL is not set, or is not an http or https URL

Writing the address into the source instead is refused, however it is spelt:

Fails with E08275: the text given to 'Networking().urlOf()' is known when the program is compiled - an endpoint in the source is configuration hard-coded: read it with 'Networking().urlFromEnv(name)', and derive other URLs from it with 'withPath', 'withPathSegment', 'withQueryParam' or 'resolve'; 'urlOf' is for URL text that arrives while the program runs

#!ek9
defines module error.e08275.bad

  defines constant
    ordersEndpoint <- "https://orders.example.com/v1"

  defines function

    hardCodedEndpoints()
      net <- Networking()
      fromLiteral <- net.urlOf("https://orders.example.com/v1")

      fromConstant <- net.urlOf(ordersEndpoint)

      base <- "https://orders.example.com"
      fromLocal <- net.urlOf(base + "/v1")
      require fromLiteral? or fromConstant? or fromLocal?

Parts: scheme(), host(), hostname() (set when the host is a name), hostAddress() (set when it is an IP literal), port(), path(), pathSegments(), queryParams() (in order, duplicates kept), fragment(), and the canonical canonicalPath() and canonicalQuery() a request signer needs.

Malformed input is rejected, never repaired: whitespace, backslashes, non-ASCII text, a user@ before the host and any scheme other than http/https all give an unset URL. IP literals in every spelling (127.0.0.1, 0x7f.1, 0177.0.0.1) are the same host. Two URLs are equal when their canonical text is equal - lower-case scheme and host, default port removed, ./.. segments resolved - and comparing them never looks anything up in DNS. For percent-encoding a single value, String has urlEncode() and urlDecode() (RFC 3986: a space is %20).

The URL policy

Because every URL enters a program through urlFromEnv or urlOf, and every other one is derived from those without changing its host, there is one point inside the program that every outbound address passes. Operations can put an allow and deny list there: set EK9_URL_POLICY to the name of a policy file and both gateways refuse what it does not permit - a refused URL is unset, exactly as a malformed one is, so the code already handles it. No capability is needed: it is an operations control on the host where the program runs, and EK9 ships the mechanism, not a threat feed.

The file has one entry per line: allow or deny, then a host (api.partner.example), a domain and all its subdomains (*.partner.example), a URL prefix (https://cdn.example.com/assets/, matched on a path-segment boundary), or an IP address or CIDR range (10.0.0.0/8). # starts a comment. Matching is on the canonical form, so a capital letter, a default port or another spelling of an IP address does not slip past an entry. A deny always wins; when the file has any allow entry, a URL must match one; and EK9_URL_POLICY_STRICT=true makes an allow entry mandatory even when the file has none. A shared host such as a code-hosting site is denied by URL prefix, never by domain. When a request is sent, a host name is resolved and refused if any address it resolves to lies in a denied range - so a name that points at the metadata endpoint or the internal network is refused like the address itself. 

# The destinations this service may use (EK9_URL_POLICY=url-policy.txt)
allow  *.partner.example          # the partner's API and every subdomain
deny   169.254.169.254/32         # the cloud metadata endpoint

This program reports three destinations from the environment: 

#!ek9
defines module site.networkTypes.urlPolicy

  defines function

    report()
      -> variableName as String
      stdout <- Stdout()
      destination <- Networking().urlFromEnv(variableName)
      if destination?
        stdout.println(`${variableName}: ${destination}`)
      else
        stdout.println(`${variableName}: refused or not set`)

  defines program

    CheckDestinations()
      report("PARTNER_URL")
      report("METADATA_URL")
      report("OTHER_URL")

Download networkTypes_url_policy.ek9

Under that policy the partner's API is allowed, the cloud metadata endpoint - the classic target of a server-side request forgery - is denied, and anything else is refused because it matches no allow entry. Each refusal is logged through the same security logger the input sanitiser uses, in the format EK9_SANITIZER_LOG_FORMAT selects (JSON by default, ECS, CEF, SIMPLE or SILENT), naming the host - never the whole URL, which may carry a token in its query - the entry that decided it and the file: 

$ EK9_URL_POLICY=url-policy.txt EK9_SANITIZER_LOG_FORMAT=SIMPLE PARTNER_URL=https://api.partner.example/v1 METADATA_URL=http://169.254.169.254/latest/meta-data/ OTHER_URL=https://elsewhere.example.org/ ek9 networkTypes_url_policy.ek9
PARTNER_URL: https://api.partner.example/v1
[URL_REFUSED] URL refused by the URL policy host=169.254.169.254 entry=deny 169.254.169.254/32 file=url-policy.txt
METADATA_URL: refused or not set
[URL_REFUSED] URL refused by the URL policy host=elsewhere.example.org entry=no allow entry matches file=url-policy.txt
OTHER_URL: refused or not set

A long-running service picks up a changed file without a restart: at most every five minutes it compares the file's modification time and swaps a changed file in whole. A file with any error is rejected as a whole and never applied in part. As the program starts, an invalid, missing or unreadable file stops it before it runs, naming the file, line and reason, with exit code 18, so it is seen at deploy time: 

allow  *.partner.example
den    169.254.169.254/32
$ EK9_URL_POLICY=broken-policy.txt PARTNER_URL=https://api.partner.example/v1 ek9 networkTypes_url_policy.ek9
ek9: URL policy could not be loaded (exit 18)
EK9_URL_POLICY: broken-policy.txt line 2: expected 'allow' or 'deny' followed by one target, found 'den    169.254.169.254/32'

If the file becomes invalid later, the last valid policy stays in force and the problem is logged, once at each five-minute check, until it is fixed. Deleting the file does not switch the policy off; unsetting the variable and restarting does.

HTTP client

HTTPClient sends a request and waits for the response - cheap on virtual threads, so there is no async form to learn. send(request) returns a Result of (HTTPClientResponse, HTTPError): any HTTP status, 404 and 500 included, is a response you read with status(); only a transport failure, a destination the URL policy refuses or a protocol error is an HTTPError, whose category() is one of DNS_RESOLUTION, CONNECT_TIMEOUT, READ_TIMEOUT, TLS_FAILURE, CONNECTION_REFUSED, TOO_MANY_REDIRECTS, PROTOCOL_ERROR or REFUSED_BY_POLICY. Nothing is thrown, and the compiler makes you check the Result before you read either side.

The client holds connections, so it is a resource a try owns and closes. A request is an immutable HTTPClientRequest: an HTTPMethod - one of GET, POST, PUT, DELETE, HEAD, PATCH, OPTIONS, written exactly - and a URL, never text; withHeader and withBody (Bytes) return a new request. A header the client sets itself (Host, Content-Length, ...) or one holding a line break is refused, giving an unset request rather than a surprise when it is sent. A response's header names are lower case, and its body is Bytes decoded only when you say how: bodyText("UTF-8") or bodyJson().

This program serves a greeting on a free port and calls it - the port is known only at run time, so the URL is text urlOf may parse: 

#!ek9
defines module site.networkTypes.httpClient

  defines service

    GreetingService for :/greeting open

      greet() as GET for :/{name}
        -> personName as String :=: PATH "name"
        <- response as HTTPResponse: (who: personName) with trait of HTTPResponse
          override status() as pure
            <- rtn as Integer: 200
          override content()
            <- rtn as Bytes: Bytes().withUtf8(`hello ${who}`)
          override contentType() as pure
            <- rtn as String: "text/plain"
          default operator ?

  defines function

    describe()
      -> result as Result of (HTTPClientResponse, HTTPError)
      stdout <- Stdout()
      if result.isOk()
        response <- result.ok()
        stdout.println(`${response.status()} ${response.header("Content-Type")}: ${response.bodyText("UTF-8")}`)
      else if result.isError()
        stdout.println(`not sent: ${result.error()}`)

    callGreeting()
      -> port as Integer
      stdout <- Stdout()
      //The port is only known at run time, so urlOf may parse this text.
      base <- Networking().urlOf(`http://127.0.0.1:${port}/greeting`)
      get <- HTTPMethod("GET")
      try
        -> client <- HTTPClient()
        describe(client.send(HTTPClientRequest(get, base.withPathSegment("Ada"))))

        //An error status is a response, not an HTTPError.
        missing <- client.send(HTTPClientRequest(get, base.resolve("/nothing")))
        if missing.isOk()
          stdout.println(`${missing.ok().status()} for /nothing`)

        //"TRACE" is not one of the seven methods, so this request is unset - an error value, not a crash.
        describe(client.send(HTTPClientRequest(HTTPMethod("TRACE"), base)))

  defines application

    GreetingApp
      register GreetingService()

  defines program

    ServeAndCall() with application of GreetingApp
      properties <- NetworkProperties(0, true)
      try
        -> httpServer <- HTTP(properties)
        caller <- (properties) is HTTPReadyHandler as function
          callGreeting(properties.port)
          server.stop()
        httpServer.serveUntilComplete(caller)

Download networkTypes_http_client.ek9

200 text/plain; charset=utf-8: hello Ada
404 for /nothing
not sent: PROTOCOL_ERROR: the request is not set

HTTPClientConfig sets how the client connects: a 10 second connect timeout and a 30 second request timeout by default (withConnectTimeout, withRequestTimeout), an explicit proxy (withProxy, never taken from the environment) and redirects. Redirects are not followed unless withFollowRedirects(true) asks - a 3xx is returned as a response, as curl does without -L. When they are followed, every hop is parsed as a URL and checked against the URL policy, at most five hops are taken, https is never downgraded to http, and Authorization and Cookie are not carried to another host. The client speaks HTTP/1.1 and always verifies TLS certificates and host names: there is no switch to turn that off. HTTPClient is a gated type: a package under capability enforcement declares org.ek9.lang::HTTPClient, which covers every type above.

Networking

Stateless utility type (always set), and the gateway through which network identities enter a program: urlFromEnv(), ipFromEnv(), cidrFromEnv() and hostnameFromEnv() read them from environment variables, never from the source, and urlOf() parses URL text that arrives at run time. It also provides well-known constants: loopbackV4(), loopbackV6(), anyV4(), anyV6(), privateA(), privateB(), privateC(), loopbackV4Net(). Composition: cidrOf(IPAddress, Integer), resolve(Hostname).

Networking is a gated type: a package under capability enforcement declares both org.ek9.lang::Networking and, because it reads the environment, org.ek9.lang::EnvVars (see Packaging).

HTTPContext

HTTP request context trait with 9 methods: principal(), isAuthenticated(), isSecure(), remoteAddress(), requestId(), header(name), contentType(), withPrincipal(identity), cookie(name). Always set. The withPrincipal() method creates an enriched copy of the context with the authenticated identity set — no mutation.

HTTP cookie record with secure defaults: secure=true, httpOnly=true, sameSite=true (Strict). The $ operator formats the cookie as a valid Set-Cookie header value with RFC 6265 percent-encoding. SameSite tri-state: true=Strict, false=Lax, unset=None.

Why EK9 does it this way. EK9 makes the secure setting the default and puts the responsibility the other way round: you must explicitly choose to be less secure.

SecurityGate

Abstract function for HTTP request authentication. Signature: (HTTPContext) → HTTPContext?. Returns an enriched context with the authenticated principal set on success, or an unset context to reject (401 Unauthorized). The default (unoverridden) behaviour rejects all requests. Use: MyGate is SecurityGate.

CORSPolicy

Abstract function for CORS origin validation. Signature: (String) → Boolean?. Returns set true = origin allowed, set false = rejected, unset = no decision (rejects by default). Use: MyPolicy is CORSPolicy.

Conclusion

EK9 has included networking types in its standard/network API from the start, because they are a key component of any modern programming language. Any networking application (UDP/TCP/HTTP) has multiple concurrent threads of execution, and networks can and do fail.

Design your applications to be as stateless and pure as possible. Where concurrent access to shared data is necessary, use a MutexLock.

Next Steps

This page used a number of programs; programs, next, explains them in more detail.