Node.js libuv and thread pool

What libuv is and why it exists

Node was built in 2009 because Ryan Dahl wanted JavaScript on the server with non-blocking I/O. He looked at Apache (one thread per connection, doesn't scale past a few thousand) and nginx (event-driven, scales to 10K+) and decided that the JS community needed an event-driven runtime.

The problem: async I/O looks completely different on every operating system. Linux has epoll. BSD and macOS have kqueue. Windows has IOCP (I/O Completion Ports). They are not just different APIs - they are different models. epoll is readiness-based: you ask "is this socket ready?" and the OS tells you yes/no. IOCP is completion-based: you start an operation and the OS notifies you when it finishes.

libuv abstracts all of this. You write code against libuv's API and it works the same on every OS. Internally, libuv uses the best async primitive on each platform. That abstraction is what lets Node be cross-platform without writing platform-specific code in every module.

The event loop in detail

libuv's event loop runs in phases, each phase has a queue, and each tick of the loop runs all the callbacks in one phase before moving to the next. The phases:

1. Timers. Callbacks for timers (setTimeout, setInterval) whose time has elapsed. The timer subsystem uses a min-heap keyed by expiry time. Each tick, libuv pops all timers whose time has passed and runs their callbacks.

2. Pending callbacks. A small bucket for deferred I/O callbacks, mostly for some TCP error conditions on certain systems. Rarely matters in application code.

3. Idle, prepare. Internal. Used by libuv handles for internal bookkeeping. You cannot schedule into these from JS.

4. Poll. The heart of the loop. Two things happen here. First, run any I/O callbacks that are ready. Second, if there are no other callbacks pending, the loop blocks in the OS poll syscall (epoll_wait, kevent, or GetQueuedCompletionStatus) for new events, up to a calculated timeout (based on the nearest timer).

5. Check. setImmediate callbacks. The name comes from the fact that they run "immediately after the current poll phase."

6. Close callbacks. Handlers for closed sockets, files, etc. (socket.on('close', ...)).

Between every callback in every phase, Node drains two queues: process.nextTick (Node-only, runs before promise microtasks) and microtasks (promise callbacks, queueMicrotask).

The poll phase is where it gets interesting

Most of Node's time is spent in the poll phase. It is the only phase that can block. Everything else either runs callbacks (which return quickly) or is a no-op.

When the poll phase has no pending I/O callbacks and the event loop has no other work, libuv blocks in the OS poll call, waiting for events. The block timeout is the time until the nearest pending timer. If a timer is set for 100ms, libuv will wait up to 100ms in epoll_wait for I/O. If I/O arrives earlier, libuv wakes up early and runs the callbacks.

This is how Node achieves zero CPU usage when idle. It is literally blocked on the OS waiting for the next event. Compare to a busy loop that polls in a while - Node never does that.

The blocking poll is also why Node's CPU usage matches actual work. A Node process serving 1000 requests/sec at low load uses 5% CPU. Same process under load uses 50% CPU. Linear scaling because there is no overhead from polling or context switching - just the work itself.

The thread pool

Some operations cannot be made async by the OS, or are async but not in a useful way:

• File system on Linux. epoll does not support regular files (it only works on sockets, pipes, devices). To do async file I/O on Linux, you have to use a thread that blocks on read() or write(). The new io_uring API (Linux 5.1+) does support proper async file I/O, and libuv added experimental support for it, but the default is still the thread pool.

• DNS lookup. getaddrinfo is the standard POSIX DNS resolver. It blocks. There is no portable async DNS API. So dns.lookup (which uses getaddrinfo) runs on the thread pool. dns.resolve (which uses the c-ares library with raw UDP) is truly async.

• Crypto. crypto.pbkdf2, crypto.scrypt, large crypto.randomBytes calls do CPU work. They run on the thread pool to avoid blocking the main thread.

• Compression. zlib.gzip, zlib.deflate, zlib.inflate (when used as async functions) run on the thread pool.

The pool has 4 threads by default. You can change this with UV_THREADPOOL_SIZE, set before Node starts. Maximum is 1024.

UV_THREADPOOL_SIZE=16 node server.js

When the pool is full, requests queue. You can fill the queue quickly with many concurrent file reads or crypto operations. The symptom in production: response latency suddenly spikes, JS profile shows the main thread is idle, and the CPU shows the four worker threads at 100%. You are pool-bound, not CPU-bound or JS-bound.

The fix is some combination of:

• Raise UV_THREADPOOL_SIZE (cheap, immediate, but takes more memory).
• Reduce concurrency at the app layer (semaphore around pool-bound ops).
• Use dns.resolve instead of dns.lookup.
• Use streams instead of buffering whole files.
• Move CPU-heavy crypto/compression to worker threads (separate from the pool).

Network I/O does not use the pool

This is the most important thing to internalize. TCP, UDP, HTTP, and TLS all use the OS's native async primitives directly. epoll/kqueue/IOCP are perfect for sockets. No thread pool. No queue. Just the event loop and the OS.

That is why Node handles thousands of concurrent connections on one thread without breaking a sweat. A typical Node server might have 5000 open sockets and 5% CPU. Each socket is just an entry in epoll's tracked set. When data arrives, epoll_wait returns and Node runs the appropriate callback.

The asymmetry between network I/O (no pool) and file I/O (pool) is the most surprising thing about Node for beginners. It looks like everything is async at the JS level, but underneath the model is very different.

process.nextTick vs setImmediate vs setTimeout(0)

Three ways to "do this later." They are not interchangeable.

process.nextTick(fn): runs after the current operation completes, before the event loop continues. Drained between every callback in every phase. Higher priority than promise microtasks. Use sparingly. Can starve I/O if abused.

setImmediate(fn): runs in the check phase of the next loop iteration. Always after the poll phase. Lower priority than I/O and timers. Use for "run after current I/O is done."

setTimeout(fn, 0): runs in the timers phase of the next loop iteration. Minimum delay is 1ms in Node (not 4ms like in browsers). Use for "run on the next tick after a brief delay."

From inside an I/O callback (you are in the poll phase), setImmediate always runs before setTimeout(0) because check phase comes right after poll. From the main module (no current phase), the order is non-deterministic because it depends on the timer's exact elapsed time vs when the loop started.

Handles and requests

libuv has two object types in its API: handles and requests.

A handle is a long-lived object that represents something the loop watches: a TCP socket, a timer, a pipe, an FS event watcher. Handles are active as long as they exist; you close them to remove from the loop.

A request is a short-lived operation: "read this file," "lookup this DNS name," "connect to this address." Requests have a callback and complete once.

You usually don't see this in JS, but if you write a Node addon you deal with uv_tcp_t (handle) and uv_read_t (request) directly. The loop reference counts handles: if there are no active handles, the loop exits. That is why Node exits when your main function returns (no active handles) but stays running if you set a timer (the timer is an active handle).

Why the loop blocks vs spins

A common confusion: doesn't an event loop just spin checking queues? No. libuv calls into the OS's epoll_wait (or equivalent) which puts the process to sleep until an event is ready. The kernel wakes Node when something happens. Until then, Node uses zero CPU.

This is the same mechanism Nginx, Redis, HAProxy, and other event-driven servers use. It is why a Node process serving 10K idle WebSocket connections uses almost no CPU.

Debugging libuv issues

Symptoms of pool exhaustion:

• Latency spikes for all requests, not just one.
• JS profile shows main thread idle most of the time.
• process.cpuUsage() shows much less than 100% of one core.
• File reads, crypto, or DNS in the request path.

Tools:

• node --prof for CPU profiling. The output is V8's tick log.
• node --inspect plus Chrome DevTools.
• clinic.js doctor walks you through diagnosis with one command. Highly recommended.
• perf on Linux gives you per-thread CPU time. You can see if the workers are pegged.

For loop lag specifically:

const start = process.hrtime.bigint();
setImmediate(() => {
  const lag = Number(process.hrtime.bigint() - start) / 1e6;
  console.log(`loop lag: ${lag}ms`);
});

Healthy lag is under 10ms. Sustained lag over 100ms means the loop is blocked - by sync code, a big synchronous parse, or microtask flooding.

When to reach for worker threads

The thread pool is for libuv internals. For your own CPU-bound JS, use worker_threads. A worker is a separate V8 isolate with its own event loop and its own thread. Communication is via MessageChannel (structured clone) or SharedArrayBuffer (shared memory).

Rule of thumb: if a single operation takes more than 10ms of pure JS CPU, offload it to a worker. Examples: image resizing in pure JS, parsing large JSON, running a regex over a megabyte of text.

Mental model

Picture a building. The main thread is the reception desk. libuv's event loop is the receptionist. The OS is the city outside (network, disk, time). The thread pool is a small back office with 4 clerks.

Every request comes to reception. Some can be answered immediately (sync JS). Some need to be sent to the city for a reply (network) - the receptionist sends the request, gets a ticket, and tells the next visitor to take a seat. When the city responds, the receptionist sees it and serves the right person.

Some requests need a clerk in the back office to do paperwork (file I/O, DNS, crypto). Only 4 clerks. If 50 requests need the back office, 4 are working and 46 are queued. Reception keeps answering other things while the queue clears.

If reception (the main thread) ever stops working - because it is solving a Sudoku puzzle (CPU-bound JS) - nothing else moves. The whole building stops.