DEV Community

Cover image for Understanding Linux File Descriptors, Epoll, and Non-Blocking I/O
DEVANSHU PATIL
DEVANSHU PATIL

Posted on AI-assisted

Understanding Linux File Descriptors, Epoll, and Non-Blocking I/O

#backend #linux #networking #performance

Understanding Linux File Descriptors, Epoll, and Non-Blocking I/O

Whenever you hear that Nginx, Redis, or Node.js can handle 100,000 concurrent client connections on a single server, the technology making it possible is the Linux epoll kernel subsystem.

In Linux, the foundational philosophy is: "Everything is a file".

Whether it is a text document on your SSD, a TCP network socket connection, an IPC pipe, or a device driver, the operating system kernel represents it using an integer known as a File Descriptor (FD).

In this deep dive, we explore how Linux manages I/O, why traditional polling mechanisms fail at scale, and how epoll provides constant-time $O(1)$ event notifications.

1. Blocking I/O vs. The C10K Problem

In traditional blocking I/O:

int bytes_read = read(socket_fd, buffer, 1024);
Enter fullscreen mode Exit fullscreen mode

If the client hasn't sent data yet, the calling thread is placed into sleep state by the OS scheduler. The thread cannot do anything else until network packets arrive.

To handle 10,000 users with blocking I/O, you need 10,000 threads. Thousands of thread stacks eat gigabytes of RAM and waste CPU cycles on context switching.

2. The Evolution of Polling: select() and poll() ($O(N)$)

To avoid thread-per-connection, Linux introduced non-blocking I/O multiplexing via select() and poll().

A single thread gives the kernel an array of 10,000 socket file descriptors and asks: "Wake me up when ANY of these sockets has data to read."

Why select() and poll() Hit a Hard Performance Wall ($O(N)$):

  1. Linear Scanning: Every time data arrives on just ONE socket, the kernel wakes up the process. The process has to loop through all 10,000 file descriptors one by one to find which single socket triggered the event!
  2. Memory Copying: On every single poll call, the user application must copy the entire array of 10,000 file descriptors into kernel space, and the kernel copies it back.

3. The Modern Solution: epoll ($O(1)$)

Introduced in Linux kernel 2.6, epoll completely redesigns the multiplexing contract.

Instead of passing an array of sockets on every call, epoll creates an in-kernel stateful context:

  1. epoll_create1(): Creates an epoll instance inside the kernel.
  2. epoll_ctl(): Registers or removes file descriptors from the monitored set (backed by a Red-Black Tree).
  3. epoll_wait(): Blocks until events occur.

When network packets arrive at the physical network interface card (NIC), the hardware interrupt handler places the active socket directly into epoll's Ready List.

When epoll_wait() returns, it returns only the exact sockets that have data ready!

Processing time is proportional only to the number of active events ($O(1)$ relative to total connections), completely eliminating the idle-connection penalty.

Top comments (0)