Work stealing
Knitting has two independent host-side scheduling features:
- Native work stealing changes how requests reach workers. Compatible multi-worker pools publish work to one shared submit region, so workers can claim tasks as they become available.
- The host doorbell changes how the host waits for responses. Supported runtimes can arm an asynchronous waiter instead of repeatedly polling the response mailbox.
Both features reduce CPU time spent waiting. A worker with no available work parks instead of spinning, and a host with no results to collect waits on the doorbell instead of polling. This keeps idle CPU use low. Multi-threading shows what that looks like in measurements.
Neither feature changes the task API. Tasks are still exported functions or
task() definitions, and calls still look like
await pool.call.name(input).
Quick start
Section titled “Quick start”import { createPool, isMain, task } from "knitting";
export const transform = task<string, string>({ f: (value) => value.toUpperCase(),});
if (isMain) { using pool = createPool({ threads: 4, host: { steal: true, doorbell: true, }, })({ transform });
console.log(await pool.call.transform("hello"));}The explicit options are useful when documenting or benchmarking a topology. For compatible multi-worker pools, native stealing is selected automatically; the task code does not need to opt in from the worker side.
Work stealing
Section titled “Work stealing”With private request lanes, the host chooses a worker and publishes the call to that worker’s mailbox. A busy worker can therefore hold queued work while a different worker is idle.
With native stealing, the pool uses a shared-submit/private-return layout:
- The host publishes a request to one shared submit region.
- A worker claims tasks from that region and runs them on its own event loop.
- Each worker keeps a private return region for its responses.
- The worker that claims a task owns its response; the host’s pending registry settles the corresponding promise.
This lets a worker that finishes early claim more available work without the host predicting which worker will be free next. Completion order remains task completion order, not submission order.
An async task returns to the worker’s event loop at its first await, and its
continuation runs on the same worker. Without a limit, a worker whose tasks are
waiting can look idle and claim more work; when those tasks resume, their
CPU-heavy continuations can bunch up on that worker while peers sit idle. Set
worker.maxAwaitingTasks to bound unresolved async tasks per worker. Once a
worker reaches the cap, queued tasks remain available for other workers to
claim.
Automatic selection
Section titled “Automatic selection”Native stealing is selected automatically when the pool has multiple workers and its configuration is compatible with the shared-submit topology. It also supports process-worker pools, subject to the same shared-memory and claimant limits. A one-worker pool keeps its ordinary private lane because there is nothing to steal from. Compiled workers, inliners, explicitly private dispatcher/balancer configurations, and pools above the 31-claimant protocol limit keep their fallback topology.
Andromeda thread pools give each worker its own request
lane by default. You can set host.steal: true to experiment with stealing,
but on Andromeda 0.1.14 the host may only collect results when a one-second
fallback timer fires. KNITTING_STEAL has no effect there because the runtime
has no process.env.
An explicit balancer, private-lane dispatcher, inliner, compiled worker, or an
unsupported worker count can change that compatibility decision. Use
host.steal: false to force private request lanes. Use host.steal: true only
when the resulting topology is supported and is what you intend to measure.
stealRegionLanes
Section titled “stealRegionLanes”The submit region has 32 slots. stealRegionLanes controls how many slots a
worker can take in one claim. With ticket, it’s the maximum batch size. With
Dekker, the slots are divided into fixed regions, and a worker takes a whole
region at once:
number of regions = 32 / stealRegionLanesUse a positive power of two. Knitting chooses the widest valid region by default, leaving a spare region alongside the worker claimants. Wider regions amortise arbitration and suit many cheap, similarly sized calls. Narrower regions expose more independent work and suit expensive or uneven calls. This setting affects throughput and load balancing, but not correctness.
stealClaim
Section titled “stealClaim”host.stealClaim chooses how workers take waiting tasks:
| Value | How workers claim tasks |
|---|---|
"ticket" | The default. A counter assigns work in the order the host publishes it. |
"dekker" | Each worker signals its intent through its own slot before claiming a region. |
using pool = createPool({ threads: 4, host: { steal: true, stealClaim: "ticket", stealRegionLanes: 1, },})({ transform });The environment variable KNITTING_STEAL_CLAIM accepts the same two values.
Setting host.stealClaim overrides the environment variable. Unknown values,
including the old "cas-mask" option, cause pool creation to throw. Dekker
needs a spare region alongside the workers, so it can limit the region width
more than ticket does.
Ticket assigns work in order, but workers can still decode or finish later claims first. Each claim gets a 64-bit ticket number that only increases. If the counter runs out, the queue closes rather than reusing a ticket number.
They also handle fatal failures differently. Dekker releases the region so another worker can finish the remaining work. With ticket, a fatal decoder or worker failure closes the shared queue. Calls still waiting for results and any new calls then reject, so shut down and replace the pool. Knitting doesn’t retry claimed work because the decoder may already have consumed some of its payload. Under either option, a task that throws or returns a rejected promise only rejects that call.
stealSingleClaimMicroseconds
Section titled “stealSingleClaimMicroseconds”In 0.1.74, workers can adjust their ticket batch size as they run tasks.
A worker normally finishes the batch it claimed, so a fast task can get stuck
behind a slow one even while another worker is idle. Set this option to let
workers take one task at a time when their recent tasks have been expensive:
using pool = createPool({ threads: 4, host: { stealClaim: "ticket", stealSingleClaimMicroseconds: 20, },})({ transform });Each worker reads the clock twice per batch to measure the average task cost.
It remembers the highest recent average and lets that estimate fall gradually
as more batches run. While the estimate meets or exceeds your threshold, the
worker takes one ticket at a time. Below it, the worker takes up to
stealRegionLanes, sharing the cost of claiming work across several cheap
tasks. Set the threshold in microseconds; the default, 0, turns adaptation
off.
This only applies to ticket pools with more than one worker. Dekker doesn’t
adjust its batches. If the ticket batch width is already 1, the worker skips
timing because it can’t take a smaller batch. A negative, NaN, or infinite
threshold causes pool creation to throw when stealing is enabled.
If your workload mixes slow and fast tasks, try 20 µs and measure the result.
The
0.1.74 README
reports better p99 latency for mixed workloads in local tests. With seven
workers running similar short tasks near the threshold, throughput fell by
about 4–7%. That’s why this is off by default: whether it helps depends on
the work you’re running.
The host doorbell
Section titled “The host doorbell”The doorbell is a host completion mechanism. It is separate from the worker loop that waits for new requests.
When the host has drained all visible responses, a polling dispatcher schedules another notification and eventually backs off with a timer. With the doorbell, the host arms an asynchronous wait on the response mailbox’s shared signal. When a worker publishes a response, it rings that signal and the host schedules another drain. If the runtime cannot arm the wait, Knitting falls back to polling.
Polling spends host CPU whether or not a response is waiting, and it spends it on the same thread that produces the work. The doorbell removes the empty checks: nothing is scheduled until a worker has something to hand back. The less loaded the pool, the larger the share of checks that were empty, which is why the doorbell matters most on a pool that is not saturated.
The doorbell is enabled by default when host.doorbell is not false and the
runtime has a completion wake path. Unsupported or denied configurations fall
back to polling.
| Runtime or topology | Completion behavior |
|---|---|
| Node.js or Bun thread workers | Atomics.waitAsync; Node can optionally use native uv_async_t |
| Deno thread workers | Thread-safe FFI callback when FFI permission is available; otherwise polling |
| Andromeda thread workers (experimental) | Atomics.waitAsync when workers have their own request lanes; with stealing, results can wait for the one-second fallback timer |
| Process workers | Process-local completion transport; otherwise polling |
| Compiled workers | Host options are unsupported by the compiled-worker path |
| Browser web workers | Polling; the browser doorbell is intentionally disabled |
Set host.doorbell: false to force polling for an apples-to-apples benchmark or
when the host is competing with workers for every CPU core. On Node thread
workers, host.nativeDoorbell: true additionally requests the optional native
uv_async_t bridge from the knitting_doorbell addon. It is off by default,
requires host.doorbell, does not apply to process workers, and silently falls
back when the prebuild or permission is unavailable.
Tuning and benchmarking
Section titled “Tuning and benchmarking”Start with the defaults. Tune stealRegionLanes or opt into
stealSingleClaimMicroseconds when task durations are highly uneven. Compare
host.doorbell: false with the supported default when host CPU or completion
latency matters.
For tasks that await input and then do substantial CPU work, try
worker: { maxAwaitingTasks: 2 }. It limits how many such tasks can be in
flight on one worker, leaving more queued work for its peers. Bundlers and
minifiers are common examples; see
Bundling and minifying. Start
at 2 and measure your workload.
Keep the topology constant while benchmarking. Do not compare a polling private-lane pool with a doorbell stealing pool and attribute the entire difference to the doorbell. These settings matter most when many calls compete for local CPU; they are unlikely to dominate a workload that mostly waits on external I/O.