PR #67288 adds an experimental
node:green_threads module to Node.js. It is a prototype. It has the
approvals it needs to land, as Stability 1.0 (Early development), but whether
it remains depends on how useful it ends up being. This post covers what it
is, how it works, how it compares with worker threads and vm contexts, and
its limitations.
What a green thread is
A green thread is a thread of execution that is scheduled by the runtime rather than by the operating system. In this implementation, each green thread is its own V8 isolate, with its own heap and its own garbage collector. It does not get an OS thread, an event loop, or a Node.js environment. Instead, all green threads share a small, process-wide pool of OS threads that run them as they have work to do.
import { GreenThread } from 'node:green_threads';
const greenThread = new GreenThread(`
onmessage = ({ data }) => postMessage(data * 2);
`);
greenThread.on('message', (value) => {
console.log(value); // 42
greenThread.terminate();
});
greenThread.postMessage(21);Inside, a green thread has the JavaScript built-ins defined by ECMAScript and
a short list of globals: postMessage() and onmessage, timers,
queueMicrotask(), a console that forwards to the host, EventTarget,
AbortController, MessageChannel, and a scoped BroadcastChannel. There
is no require(), no process, no file system, no network. A green thread
can only do what the host gives it the ability to do.
How they work
Every message, timer callback, and evaluation that a green thread receives is a task. The tasks of one green thread run one at a time and in order, but not necessarily on the same OS thread. When a task is queued, the green thread is scheduled onto whichever pool thread is free. When the task finishes, the pool thread moves on to the next green thread that has work. A green thread that is waiting for something costs memory, but no thread.
The pool has as many threads as os.availableParallelism(), up to 8 by
default, or as many as the NODE_GREEN_THREADS_POOL_SIZE environment
variable sets. At most that many green threads run JavaScript at the same
time. Thousands of them can exist at once.
Tasks are not preempted. A task that runs for a long time keeps its pool thread for that long, unless it hits a CPU time limit (more on those below).
Talking to the host
There are a few ways to get data in and out.
Messages work like they do with workers: postMessage() on either side,
with structured cloning and transferable ArrayBuffers. MessagePorts are
the same ports that node:worker_threads uses, so a port can connect a green
thread to its host, to another green thread, or to a worker.
The host can also pass an env object. Functions in it become stubs that
the green thread can call, and the calls return promises. With
type: 'module', the green thread's source is an ES module, and its exports
come back to the host the same way. Modules that it imports come from the
host too, either as a bundle or from resolve() and load() hooks, since the
green thread can't read files.
import { GreenThread } from 'node:green_threads';
const plugin = new GreenThread(`
import { slugify } from './util.js';
export async function transform(text) {
const prefix = await env.config('prefix');
env.log('transforming', text);
return prefix + slugify(text);
}
`, {
type: 'module',
filename: 'file:///plugins/slug/index.js',
modules: {
'file:///plugins/slug/util.js':
`export const slugify = (s) => s.toLowerCase().replaceAll(' ', '-');`,
},
env: {
async config(key) {
return { prefix: 'post-' }[key];
},
log(...args) {
console.log('[slug]', ...args);
},
},
resourceLimits: { maxTaskCpuTimeMs: 100, maxMemoryMb: 32 },
});
const { transform } = await plugin.exports;
console.log(await transform('Hello World')); // post-hello-world
await plugin.terminate();Errors thrown on one side reach the other as real Error objects that keep
their class, message, stack, code, and cause. Objects that extend
RpcTarget and AbortSignals can be passed in both directions as well.
Limits
Each green thread can have its own limits on CPU time and memory. The CPU
limits count CPU time, not wall time, so a green thread that is waiting on a
timer or a host call isn't charged for it. maxTaskCpuTimeMs applies to each
task and maxCpuTimeMs to all of them together. maxMemoryMb covers the JS
heap and ArrayBuffers together.
import { GreenThread } from 'node:green_threads';
const spin = new GreenThread('onmessage = () => { for (;;); };', {
resourceLimits: { maxTaskCpuTimeMs: 50 },
});
spin.on('error', (error) => console.log(error.code));
// ERR_GREEN_THREAD_CPU_LIMIT
spin.postMessage('go');
const hog = new GreenThread(`
onmessage = () => {
const buffers = [];
try {
for (;;) buffers.push(new ArrayBuffer(1024 * 1024));
} catch (error) {
postMessage(\`\${error.name} after \${buffers.length} MB\`);
}
};
`, { resourceLimits: { maxMemoryMb: 32 } });
hog.on('message', (value) => {
console.log(value); // RangeError after 29 MB
hog.terminate();
});
hog.postMessage('go');A green thread that reaches a limit is stopped and emits an 'error' event.
Nothing else in the process is affected.
Warm pools
Creating a green thread means creating an isolate, which takes a few hundred
microseconds. setWarmPoolSize() prepares isolates ahead of time, so that
starting a green thread takes one out of the pool instead:
import { GreenThread, setWarmPoolSize } from 'node:green_threads';
// Prepare 100 isolates ahead of time.
await setWarmPoolSize(100);
const start = performance.now();
const greenThreads = [];
for (let i = 0; i < 100; i++) {
greenThreads.push(new GreenThread(`postMessage(${i});`));
}
await Promise.all(greenThreads.map((t) => new Promise((resolve) => {
t.once('message', resolve);
})));
console.log(`${(performance.now() - start).toFixed(1)} ms for 100`);
// 1.6 ms for 100Compared with worker threads and vm contexts
Node.js already has two ways to run JavaScript separately from the main thread's code.
A worker thread is a full Node.js environment on its own OS thread, with its
own isolate and event loop. It can do anything the main thread can,
including require('fs'), and it can block.
A vm context is a separate global object inside an existing isolate. It is
cheap, but it shares the heap and the garbage collector of the isolate it
lives in, and the vm documentation says plainly that it is not a security
mechanism. Its timeout option can stop synchronous code that runs too long,
but it can't limit memory. A context that fills the heap takes the whole
isolate down with it.
Green threads sit between the two. Like a worker, a green thread has an
isolate of its own, so it can't reach the host's objects, its heap is
separate, and it can be stopped without affecting anything else. Like a vm
context, it is cheap enough to have thousands of. Unlike either, it has no
Node.js APIs at all, and it has CPU time and memory limits built in.
A note on isolation: an isolate is a much stronger boundary than a vm
context, but green threads still share a process with the host and with each
other. I would not call this a security sandbox for hostile code on its own.
Numbers
All of these come from the benchmark scripts I used during development, run on an 88 CPU Linux x64 machine, with a release build of the PR branch and the default pool of 8 threads. Each measurement runs in a fresh process. Memory is PSS, measured after the instances have started and are idle. The numbers vary by a few percent between runs, and by more on other machines, so read them as rough proportions rather than exact figures.
Starting 1,000 idle instances:
| Memory per instance | Started per second | |
|---|---|---|
| Worker | 6.2 MB | ~950 |
| Green thread | 1.4 MB | ~3,700 |
| Green thread, from a warm pool | 1.4 MB | ~83,000 |
vm context, spread over 8 workers | 0.25 MB | ~15,500 |
vm context, created ahead of time | 0.25 MB | ~120,000 |
A green thread uses about 4.5x less memory than a worker. A vm context uses
about 5.5x less than a green thread, and is faster to create.
Raw message speed, sequential round trips with one instance:
| Round trips per second | Latency | |
|---|---|---|
| Worker | ~54,000 | ~18.5 µs |
| Green thread | ~50,000 | ~20 µs |
Workers are slightly faster per message.
CPU-bound work, 2,000 calls to fib(27):
| Tasks per second | |
|---|---|
| 8 workers | ~5,000 |
| 8 green threads, pool of 8 | ~4,800 |
| 88 workers | ~22,000 |
| 88 green threads, pool of 88 | ~21,000 |
At the same parallelism, the same code runs at about the same speed. Green threads don't make computation faster. They can't use more cores than the pool has threads.
1,000 tenants, each with code and state of its own, serving 50,000 requests spread over all of them, 64 at a time:
| Requests per second | |
|---|---|
| A worker per tenant | ~160,000 |
| A green thread per tenant | ~195,000 |
The worker version runs 1,000 OS threads. The green thread version runs 8.
Against a pool of 8 workers that runs each tenant in a vm context, in a
separate benchmark with the same workload:
| Requests per second | p50 | p99 | |
|---|---|---|---|
| Green threads | ~181,000 | 0.33 ms | 1.32 ms |
vm pool | ~180,000 | 0.36 ms | 0.76 ms |
vm pool with timeout: 50 | ~79,000 | 0.64 ms | 3.05 ms |
Without a timeout, throughput is the same, and the vm pool has better tail
latency. A host that runs untrusted tenants needs the timeout option,
though, and with it the vm pool serves less than half as many requests.
The bigger difference is what happens when one of 100 tenants misbehaves:
| One bad tenant out of 100 | Green threads | vm pool of 8 |
|---|---|---|
| Synchronous infinite loop | Stopped after 51 ms. Slowest other request: 3 ms. | Stopped by timeout after 51 ms. Tenants on the same worker wait up to 51 ms. |
| Loop in microtasks, after returning | Stopped after 51 ms. | timeout doesn't apply. The worker has to be killed (after 101 ms here), and 13 tenants go with it. |
| Fills its heap | Stopped by maxMemoryMb. The other 99 keep answering. | The worker runs out of memory. 13 tenants go with it. |
The microtask case can be fixed in the vm pool with
microtaskMode: 'afterEvaluate', and then timeout stops it too. The heap
case can't: the contexts in a worker share its heap, so the only way to keep
one tenant from taking others down with it is a worker per tenant.
One thing in the vm pool's favor: a timeout fails only the request, and
the tenant keeps its state. A green thread that hits a limit is gone.
What they are good for
Based on the above, green threads make sense for:
- Hosting many small, mostly idle units of code that each need isolation from the others: multi-tenant handlers, user-defined functions, webhooks.
- Plugin and extension systems, where the plugin should get only the capabilities the host passes it, with CPU and memory limits.
- User-supplied scripts, rules, and transforms.
- Anything where startup time matters and the code can be prepared for ahead of time with a warm pool.
Workers remain the better choice for code that needs Node.js APIs, for npm packages that use them, for code that blocks, and for a few long-lived units where the cost per instance doesn't matter.
vm contexts remain the cheapest option when the code is trusted, or when
losing every tenant in a worker to one bad one is acceptable.
Limitations
- No Node.js APIs. Not
fs, notnet, and at the moment not evenURL,TextEncoder,structuredClone(), orfetch(). npm packages that use Node.js APIs won't run, and any others have to be passed in by the host as modules. What the right set of APIs is, is one of the open questions. - No blocking.
Atomics.wait()throws in green threads, because it would block a pool thread. Code that spins on aSharedArrayBufferinstead can still keep pool threads busy, and if enough green threads do that while waiting for one that hasn't been scheduled, they wait forever.Atomics.waitAsync()gives the thread back while waiting and avoids the problem. For code the host doesn't control,setStallPolicy()detects these stalls and breaks them by adding pool threads or terminating the green thread that has been running the longest. It can't detect deadlocks that don't depend on the pool, such as two green threads that each hold a lock the other one wants. A CPU time limit stops those. - No preemption. A long task holds its pool thread, and the timers of other green threads that run on that thread wait for it.
- No debugger. There is no inspector integration, so DevTools can't attach to a green thread.
- Higher memory and slower creation than
vmcontexts, and slightly slower messaging than workers. - Parallel computation is limited to the size of the pool.
- In rare cases, V8 compiles a loop in a way that ignores termination requests. A green thread that doesn't stop within a second is abandoned, and Node.js tries to stop the pool thread that runs it, which is only possible on Linux x64 and arm64. If that doesn't work within 10 seconds, Node.js aborts the process.
Trying it out
The code is not in a release. To try it, build the PR branch:
git clone https://github.com/nodejs/node
cd node
git fetch origin pull/67288/head:green-threads
git checkout green-threads
./configure --ninja
make -j16
./node --experimental-green-threads my-app.mjsThe documentation in the branch covers the full API.
Once it lands, the next step is to decide whether it is worth keeping, and if it is, which additional APIs green threads need. If you have a use case for them, or one that they don't fit, please try it out and leave feedback on the design and the use cases on the PR.