Quick Start
Define your tasks, create a pool, and call them like async functions. The pool closes automatically when you are done. This guide walks through each step.
Introduction
Section titled “Introduction”Knitting gives workers a function-call API with low communication overhead. If you are new to workers, start with these terms.
Quick definitions
Section titled “Quick definitions”- Task: a function the workers can run. An exported function already counts
as one; wrap it in
task({ f })when you want options like timeouts or aborts. call.*(): runs a task on the pool and returns aPromisewith the result.isMain:trueon the host,falseinside a worker. Workers re-import your module, so anything that should happen once — creating the pool, starting a server — belongs behind this check.createPool(): starts the workers and returns a pool with a typedcallobject and ashutdown()method. The pool is disposable, sousingcan close it.- Host ↔ Worker: the host is the process that creates the pool. The workers are the threads (or processes) that run the tasks.
Examples
Section titled “Examples”These four examples cover a single task, parallel calls, multiple tasks, and task options.
import { createPool, isMain } from "knitting";
// A task is just an exported function the workers can run.export const greet = (name: string) => `hello ${name}`;
if (isMain) { // `using` shuts the pool down automatically when this block ends. using pool = createPool({ threads: 1 })({ greet });
console.log(await pool.call.greet("knitting")); // hello knitting}import { createPool, isMain } from "knitting";
export const hello = () => "hello ";export const world = (prefix: string) => `${prefix}world!`;
if (isMain) { using pool = createPool({ threads: 2 })({ hello, world });
// call.hello() returns a promise; Knitting resolves it before world runs. const lines = await Promise.all( Array.from({ length: 3 }, () => pool.call.world(pool.call.hello())), );
console.log(lines.join(" ")); // hello world! hello world! hello world!}import { createPool, isMain } from "knitting";
// Several tasks share one pool. Calls are promises, so you can chain them.export const double = (n: number) => n * 2;export const square = (n: number) => n * n;
if (isMain) { using pool = createPool({ threads: 2 })({ double, square });
const results = await Promise.all( [1, 2, 3, 4, 5].map(async (n) => pool.call.square(await pool.call.double(n))), );
console.log(results); // [4, 16, 36, 64, 100]}import { createPool, isMain, task } from "knitting";
// Wrap a function with task() when you want options like a timeout.// This call is too slow, so it falls back to the default instead of hanging.export const slow = task({ timeout: { time: 100, default: "timed out" }, f: async (name: string) => { await new Promise((resolve) => setTimeout(resolve, 1_000)); return `hello ${name}`; },});
if (isMain) { using pool = createPool({ threads: 1 })({ slow });
console.log(await pool.call.slow("knitting")); // timed out}Build it step by step
Section titled “Build it step by step”-
Import what you need:
import { createPool, isMain } from "knitting"; -
Export your tasks at module scope. Workers find them by name, so they have to be reachable from the top level of the file:
export const square = (n: number) => n * n;export const greet = (name: string) => `hello ${name}`; -
Create the pool behind
isMain. Workers re-import this module, and without the guard every one of them would try to start a pool of its own:if (isMain) {using pool = createPool({ threads: 2 })({ square, greet });}usingcloses the pool when the block ends, so there is nothing to clean up. -
Call the tasks. They hand back ordinary promises, so
Promise.allbatches them:if (isMain) {using pool = createPool({ threads: 2 })({ square, greet });const [n, message] = await Promise.all([pool.call.square(8),pool.call.greet("knitting"),]);console.log({ n, message }); // { n: 64, message: "hello knitting" }} -
Shut down when you are done.
With
using, the workers stop when the block ends and the process can exit. Callshutdown()yourself to close the pool earlier, or if your runtime does not supportusing:const pool = createPool({ threads: 2 })({ square, greet });try {console.log(await pool.call.square(8));} finally {await pool.shutdown();}
A task can make its own pool
Section titled “A task can make its own pool”For a short script with one task, chain .createPool() onto the task definition:
import { isMain, task } from "knitting";
export const double = task({ f: (n: number) => n * 2,}).createPool({ threads: 2 });
if (isMain) { try { console.log(await double.call(21)); // 42 } finally { await double.shutdown(); }}Good habits
Section titled “Good habits”These habits help keep worker startup fast and make common problems easier to avoid.
Keep tasks in their own module(s)
Section titled “Keep tasks in their own module(s)”Workers load the task module and its imports. Keeping that module small helps them start quickly without loading unrelated application code.
- package.json
- deno.json
Directorysrc
Directoryknitting
- database.ts
- img_parsing.ts
- jwt.ts
Directoryapp/
- …
Directorypages/
- …
Promise inputs are awaited on the host
Section titled “Promise inputs are awaited on the host”call.*() accepts Promise<supported> inputs. Knitting resolves them on the
host before dispatch, so unresolved promise state never crosses the thread
boundary. Request handlers get this for free — hand the call a body you have
not read yet:
app.post("/validate", async (c) => { const result = await pool.call.validate(c.req.text()); return c.json(result);});If that promise rejects, the call rejects on the host and the worker never runs.
Pick the cheapest payload that fits
Section titled “Pick the cheapest payload that fits”Small, flat payloads are cheaper to transfer. Prefer numbers, booleans, and
short strings when they fit your data, then typed arrays, Buffer, or compact JSON. When you have bytes plus some
metadata to describe them, use Envelope. See
Supported payloads.
Start strict, open up later
Section titled “Start strict, open up later”Worker permissions start restricted. Sensitive paths like .env, .git,
~/.ssh and /etc are blocked, and writes to node_modules are denied. Grant
only the access your tasks need. See Permissions.
Footguns
Section titled “Footguns”Almost everything here follows from one fact: workers re-import the module that defines your tasks.
- One argument per task.
pool.call.add(a, b)won’t work — pass a tuple or object:pool.call.add([a, b])for([a, b]) => a + b. - Guard host code with
isMain. Without it, pool creation (and any other host-only code) re-runs inside every worker. - Top-level imports run in every worker.
importis hoisted, so it executes before anyisMaincheck. Keep tasks in their own lean module so workers don’t load your whole server framework. - Export your tasks. An unexported
task()/importTask()is invisible to the worker loader, so the call just hangs — no handler is ever registered. importTasktargets are plain functions, nottask()wrappers (that throws aTypeError). Puttimeout/abortSignaloptions on theimportTaskcall instead.- Worker
console.*is silent by default in strict mode. Passpermission: { console: true }to surface worker logs. - Can’t tell what the pool is doing? Pass
debug: truetocreatePool, or setKNITTING_DEBUG=*. Setup, import and lifecycle diagnostics go to stderr, each line tagged with its worker and a millisecond timer. If that is too much, name the parts you care about:debug: { host: true, imports: true }. When diagnostics are disabled, the logger module is not imported. - Only supported payloads cross the boundary.
Map,Set, class instances, and functions are rejected — see Payloads. - Dynamic payloads cap at ~8 MiB by default; raise
payload.maxPayloadBytes(andpayload.payloadMaxByteLength) for larger ones.
Where to go next
Section titled “Where to go next”- Defining tasks —
task(),importTask(), timeouts, and aborts. - Creating pools — threads, balancers, and shutdown.
- Payloads — what crosses the boundary, and
Envelope. - Performance and the inliner — when to let the host run some work too.
For stronger isolation, workers can each run as a
separate process, inside a bwrap sandbox or a
container, when threads do not provide enough isolation. Large buffers can skip the
copy entirely with ProcessSharedBuffer, which puts
the bytes in shared memory instead of sending them.