epoll vs. kqueue: How Operating Systems Learned to Wait for I/O Efficiently

From Blocking to Efficiency: The Evolution of I/O Multiplexing

Modern servers routinely handle tens of thousands of simultaneous network connections. When a process needs to monitor a large number of file descriptors (sockets, pipes, files), figuring out which ones are ready for reading or writing becomes a core problem the operating system must solve. epoll (Linux) and kqueue (BSD/macOS) are the high-performance answers these operating systems have given.

At their core, both are event notification mechanisms — they let the kernel watch a set of file descriptors on the application's behalf, notifying the application when an event occurs. This eliminates the waste of the application constantly polling on its own. Understanding these two mechanisms is a critical step toward understanding the internals of high-concurrency systems like Nginx, Redis, and Node.js.

The Legacy Baggage of select and poll

Before epoll and kqueue, Unix systems provided select and poll. Their approach was straightforward: the application packages all the file descriptors it cares about and passes them to the kernel, which checks them one by one and returns which ones are ready.

The problem is scalability. select uses a fixed-size bitmap, typically capped at just 1,024 descriptors; every call requires copying the entire descriptor set from user space to kernel space, and after it returns the application must linearly scan for the ready ones. poll removes the count limit but still can't escape the fate of "passing the full list on every call + O(n) kernel traversal."

Once connections reach the tens of thousands, this O(n) overhead grows linearly with connection count — even if most connections have no activity at that moment. This is the famous C10K problem: how to handle ten thousand concurrent connections on a single machine.

The "C10K problem" was formally articulated by engineer Dan Kegel in 1999. The core tension was that under the traditional one-thread/one-process model, ten thousand concurrent connections meant ten thousand system threads, and the memory footprint (default stack size is typically 1–8 MB) plus context-switching overhead would quickly exhaust server resources. Even with thread pools, the O(n) traversal of select/poll remained a bottleneck — if a server maintains 10,000 connections but only 10 are active at any moment, the kernel still has to traverse all 10,000 descriptors to find those 10 ready ones. As broadband internet became widespread and long-lived connections (instant messaging, online gaming) proliferated, this tension came to a head in the early 2000s, directly driving the creation of epoll and kqueue and inspiring event-driven architectures like Nginx and Node.js.

epoll: Linux's Solution

epoll redesigns the entire flow with three system calls:

• epoll_create: Creates an epoll instance in the kernel and returns a file descriptor.
• epoll_ctl: Adds, modifies, or removes descriptors to watch in that instance — only needs to be called when the watch list changes.
• epoll_wait: Waits for events and returns only the descriptors that are already ready.

The key improvement is state persistence in the kernel. The application no longer needs to pass the full list to the kernel on every call. Internally, the kernel uses a red-black tree to manage registered descriptors and maintains a ready queue. When a descriptor becomes ready, the kernel attaches it to the ready queue via a callback, and epoll_wait simply returns that queue directly — complexity approaches O(1), tied to the number of active connections rather than the total number of connections.

Level-Triggered vs. Edge-Triggered

epoll offers two operating modes. Level-Triggered (LT) is the default: as long as a descriptor still has data to read, every epoll_wait call will keep notifying you. It's more forgiving to program against. Edge-Triggered (ET) only notifies once at the moment the state changes, requiring the application to drain all data in one go (typically using a non-blocking I/O loop until EAGAIN). It delivers higher performance but is easier to get wrong.

The choice between trigger modes has a profound impact on the programming model. LT behaves like select/poll — high fault tolerance, and a missed read won't cause the event to be lost forever. It's ideal for rapid development or migrating legacy code. ET requires the developer to loop with non-blocking I/O after receiving a notification, reading until EAGAIN (signaling the kernel buffer is empty); if only part of the data is read and the code returns to wait for the next notification, the remaining data will never trigger another event, causing the connection to "freeze." Nginx uses ET mode by default to reduce the number of epoll_wait wakeups — each worker process loops continuously on read events until the buffer is exhausted, which is one reason for its high throughput. In practice, ET mode also requires special handling of the accept thundering herd problem, where multiple workers are woken up simultaneously to compete for new connections.

kqueue: BSD's General-Purpose Event Framework

kqueue originated in FreeBSD and also appears on macOS. Its design is more general and abstract than epoll. The core consists of two system calls: kqueue creates the event queue, and kevent handles both registering events and retrieving them.

What makes kqueue truly powerful is that it goes beyond monitoring socket reads and writes — it can uniformly handle filesystem changes, process exits, signals, timers, and more. Developers describe "what type of event to watch for" and "what behavior they want when it fires" through a unified kevent struct, consolidating what would otherwise be scattered notification mechanisms under a single API. By comparison, epoll focuses more narrowly on file descriptor I/O readiness notifications — a tighter scope, but also a more focused one.

The kevent struct uses the filter field to distinguish event types: EVFILT_READ/EVFILT_WRITE for socket read/write readiness, EVFILT_VNODE for filesystem changes (the BSD alternative to inotify), EVFILT_PROC for process state monitoring, EVFILT_SIGNAL for signal delivery, and EVFILT_TIMER for high-precision timers. This unified abstraction lets a developer wait on network I/O, file changes, and timeouts all in a single kevent call, avoiding the complexity of combining epoll, inotify, signalfd, timerfd, and other separate APIs as required on Linux. On macOS, FSEvents, Grand Central Dispatch (GCD), and libuv (the foundation of Node.js) all build on kqueue for their BSD/macOS implementations.

The Shared Philosophy

Despite their different APIs, epoll and kqueue follow the same philosophy:

• Register once, wait many times: Decouple maintaining the watch list from waiting for events, avoiding redundant data passing.
• Return only active events: Tie overhead to the number of events that actually occurred, not the total number of watched descriptors.
• Kernel-initiated notification: Leverage the kernel's internal callback/event-driven mechanism to transform linear polling into event push delivery.

This philosophy is what makes it possible to support millions of concurrent connections on a single machine, and it forms the foundational layer of virtually every high-performance networking framework and async runtime today — including libuv, libev, and Netty's epoll transport.

Conclusion

epoll and kqueue may appear to be just a handful of system calls, but they represent a deep answer from the operating system to a fundamental question: how do you wait efficiently? The evolution from the bulk polling of select/poll to event-driven readiness notification didn't just solve the C10K problem — it defined the paradigm for modern high-concurrency programming. For any developer who wants to deeply understand network programming, async I/O, or server architecture, mastering these principles is an essential part of the curriculum.