E08251: mutation of shared state must happen inside MutexLock.enter(...) — wrap the write in a MutexKey body

Category: E08xxx Pre-IR checks (flow analysis) · Reported in phase: PRE_IR_CHECKS · Classification: SHARED_STATE_MUTATION_OUTSIDE_LOCK · All error codes

What went wrong

Code that runs on another thread changed state that another thread can also reach, outside a MutexLock.

Code runs on another thread when the runtime starts it there - a "concurrent entry point" - or when such code calls it:

  • an HTTP service operation (each request has its own thread);
  • a handler given to TCP accept or acceptUntilComplete, the ready handler of acceptUntilComplete or HTTP serveUntilComplete, and a handler given to Signals register (the signal dispatcher's thread)
  • in fact every TCPHandler, TCPReadyHandler, TCPReadyObserver, HTTPReadyHandler and SignalHandler function, however it reaches the call (a parameter, a field, a function's result): it exists to run on the runtime's threads;
  • every function in a stream pipeline that contains 'async': the workers, and every other stage too - a 'map' or 'filter' after 'async' runs on the submitting thread WHILE the workers run;
  • every method and function any of these call, directly or not, and the advice of every aspect registered with a component they call (the proxy runs it on the caller's thread).

A program that registers a signal handler runs its OWN body beside the signal dispatcher for the rest of its life. So in such a program - its body, and every function and method it calls - what a signal handler can reach is shared state too: a field of a type the handler reaches (through what it captures or injects), written in that type's own methods or through an object of the type, and the very value a handler captured, changed in place. Before 'Signals register' or after: the same.

State is shared when another thread can reach it:

  • a field of a component or service, or of an aspect registered with one: one instance serves every request, so 'this.count: count + 1' in a component method or an aspect's advice races;
  • a variable the concurrent code CAPTURED: '(shared) is task as function' closes over the very object the creating thread still holds, so 'shared.total: 7' or 'shared.add(x)' changes it under that thread;
  • a field of any object whose TYPE a shared value can reach - a field type of a component, a captured value's type, a lock's payload type: changing such an object through one of its methods is the same write, made inside the method;
  • a local naming any of these ('counter <- shared.total'), when it is mutated in place.

A write is any change: an assignment to a field, or an in-place operator - counter++ and counter-- (Integer's ++ changes the object, exactly as += does), += -= *= /= :=: :^: :~: - or streaming into a target (cat items > shared.list, >> shared.list, | tee in shared.list).

Not shared, and so allowed:

  • an object the concurrent code makes for itself, of a type no shared value can reach (a CONFINED type): a request object built per connection, a tally each worker builds and returns - its own methods may change its fields freely; a type reached from shared state only through a MutexLock's payload counts the same here (an Order being built before it is handed over to the lock), since such an object is either under the lock or still its builder's;
  • a copy (n <- Integer(shared.total)): it is a new object;
  • re-pointing a local (counter := 5): only the local changes;
  • writing to Stdout or Stderr (each line is written whole; the order of lines from different threads is not guaranteed);
  • a constructor body: the object has not escaped yet (E08250).

Rationale

EK9 guarantees at compile time that every change to state two threads can reach happens inside MutexLock.enter(...). There is no "it is only a counter" or "the next read will see it": without the lock nothing orders one thread's write before another's read (the JVM may never show the reader the new value), and natively the two threads also change the object's reference count without coordination, which can free an object still in use.

The rule is decided by TYPE. If any value of a type can be reached from shared state, a write to that type's fields from concurrent code needs the lock, wherever the object came from; if none can, the type is confined and every object of it belongs to the thread that made it. A type used both per request and inside a component is therefore shared everywhere: give the per-request use its own type, or copy.

How to fix it

  1. Keep the shared value in a MutexLock and change it inside a MutexKey body (the [EXAMPLE] below). A MutexLock is built only in a field initialiser (E08256), so put it in a component or a small class: 'level as MutexLock of Integer: MutexLock(Integer(0))', with methods that change it inside a MutexKey and a 'value() as pure' that copies it out under the lock. A signal handler that raises a debug level the program reads is exactly this.
  2. If each thread can have its own, build it inside the concurrent code instead of capturing it, and hand the result back: an async worker RETURNS its value and the pipeline delivers it to the submitting thread (the second example). Do not accumulate into a captured list or total.
  3. If the value never changes after it is made, initialise it at its declaration or in the constructor and do not write it again: reading is not a race.

Common causes

  • A service or component caching per-request data in a field: fine under one user, a race under load.
  • A signal handler adjusting a captured setting (a debug level, a flag) that the program body reads.
  • A program that registers a signal handler and then - or before - changes, outside a lock, the settings object the handler reads: build it complete in its constructor, or hold it in a MutexLock.
  • Async workers adding to a captured list or total instead of returning their values.
  • A 'map' or 'filter' stage after 'async' updating a captured counter - it runs while the workers do.
  • Capturing an object from the program body into a handler or worker and calling its mutating methods.
  • One type used both per request and inside a component: the component's use makes it shared for every caller.

See also

  • E08250: this-escape from constructor (prevents partially- initialised state being observed concurrently)
  • E08256: MutexLock construction outside field initialiser
  • E08257: MutexLock argument must be a fresh value
  • E08258: MutexLock payload exposed

Examples

Every example is compiled by the EK9 compiler: the first fails exactly as captioned, the others fix it.

Fails with E08251: mutation of shared state must happen inside MutexLock.enter(...) — wrap the write in a MutexKey body

#!ek9
defines module error.e08251.bad

  defines component

    Counter as abstract
      bump() as abstract
      default operator ?

    UnsafeCounter extends Counter
      count <- 0

      override bump()
        count: count + 1

      default operator ?

  defines service

    Counters for :/counters open

      countRequests() as GET for :/count
        <- response as HTTPResponse: () with trait of HTTPResponse
          override status() as pure
            <- rtn as Integer: 200
          override content()
            <- rtn as Bytes: Bytes().withUtf8("ok")
          override contentType() as pure
            <- rtn as String: "text/plain"
          default operator ?

        counter as Counter!
        counter.bump()

  defines application

    CounterApp
      register UnsafeCounter() as Counter
      register Counters()

Fixed: Hold the shared state in a MutexLock and change it inside a MutexKey

#!ek9
defines module error.e08251.good

  defines component

    Counter as abstract
      bump() as abstract
      default operator ?

    SafeCounter extends Counter
      count as MutexLock of Integer: MutexLock(Integer(0))

      override bump()
        bumper <- () is MutexKey of Integer as function
          lockedItem++
        require count.enter(bumper)

      default operator ?

  defines service

    Counters for :/counters open

      countRequests() as GET for :/count
        <- response as HTTPResponse: () with trait of HTTPResponse
          override status() as pure
            <- rtn as Integer: 200
          override content()
            <- rtn as Bytes: Bytes().withUtf8("ok")
          override contentType() as pure
            <- rtn as String: "text/plain"
          default operator ?
        counter as Counter!
        counter.bump()

  defines application

    CounterApp
      register SafeCounter() as Counter
      register Counters()

Fixed: Or build the value in the concurrent code and return it, instead of changing one you captured

#!ek9
defines module error.e08251.good.confined

  defines class

    Tally
      count as Integer?

      default private Tally()

      Tally()
        -> start as Integer
        count: start

      add()
        -> amount as Integer
        count += amount

      total() as pure
        <- rtn as Integer: count

      override operator ? as pure
        <- rtn as Boolean: count?

  defines function

    work() as abstract
      <- rtn as Integer?

  defines program

    ConfinedTallies()
      stdout <- Stdout()

      small <- () is work as function
        tally <- Tally(0)
        tally.add(2)
        tally.add(3)
        rtn: tally.total()

      large <- () is work as function
        tally <- Tally(100)
        tally.add(50)
        rtn: tally.total()

      totals <- cat [small, large] | async | sort | collect as List of Integer
      stdout.println($totals)

Questions and answers

These Q&A entries show E08251 and its fix:

Found a problem?

If EK9 reported E08251 for a program you think is valid, or did not report it where it should have, use the compiler form with the smallest program that shows it. If this page is wrong or unclear, use the documentation form. Both are at github.com/ek9lang/ek9-issues.