Building Container Networking From Scratch: A Deep Dive Into Linux Bridge and veth

Why Container Networking Is Worth Understanding Deeply

After running docker run to start a container, it can access the internet, communicate with other containers, and even expose port services externally—all of which seems to happen automatically. But behind this "magic" is a whole set of networking primitives provided by the Linux kernel, quietly doing the heavy lifting.

Truly understanding the underlying mechanics of container networking not only helps you quickly troubleshoot common issues like "container can't reach the internet" or "port mapping doesn't work," but is also essential groundwork for mastering the Kubernetes networking model, CNI plugins, and even service meshes.

Using "building a Linux Bridge network from scratch by hand" as its main thread, this article dissects the core building blocks of container networking layer by layer, giving you a clear view of the complete truth behind Docker's default bridge mode.

The Three Core Building Blocks of Container Networking

The isolation and connectivity of container networking doesn't rely on any brand-new technology. Instead, it cleverly combines three capabilities that have long matured in the Linux kernel. Understanding these three foundational components is a prerequisite for grasping the entire system.

Network Namespace

The network namespace is the cornerstone of container network isolation. Each namespace has a completely independent network stack—including its own network interfaces, routing table, iptables rules, and port space. When Docker creates a container, it assigns it a dedicated network namespace, so that the container's eth0 is entirely isolated from the host's network interfaces.

The network namespace is one of the seven isolation types in the Linux Namespace mechanism (the other six being Mount, UTS, IPC, PID, User, and Cgroup), and it matured after Linux kernel 3.0. Its isolation is thorough: each namespace has its own /proc/net directory, its own socket table, and its own ARP cache. The kernel manages a namespace's lifecycle via reference counting through file descriptors under /proc/<pid>/ns/net. This is the fundamental reason why running netstat or ss inside a container only shows that container's network connections.

Extended Background: The Evolution of Linux Namespaces The Linux Namespace mechanism was not a one-shot design, but the result of over a decade of gradual evolution. The earliest Mount Namespace was introduced in 2002 with kernel 2.4.19, while the network namespace only became usable around 2009 in the 2.6.24–2.6.29 kernel series, reaching production-grade stability after 3.0. This incremental evolution reflects the Linux community's consistent philosophy of "minimizing kernel changes and reusing existing abstractions." The seven namespace types are mutually orthogonal and can be combined independently, giving container runtimes enormous flexibility—for example, Docker shares the UTS namespace by default so that containers can see the host name, while Kubernetes' hostNetwork mode directly reuses the host's network namespace, entirely bypassing the virtual network layer to achieve the lowest latency.

When operating manually, you can create a namespace with ip netns add <name> and execute commands within it using ip netns exec <name> <command>. This is the first step of "building from scratch": carving out an independent network territory for the "container."

Virtual Ethernet Pair (veth pair)

With an isolated namespace in place, the next question is: how do we let it communicate with the outside world? The answer is the veth pair. A veth (Virtual Ethernet) device always comes in pairs—you can think of it as the two ends of a virtual network cable: data entering one end must exit from the other.

The typical approach is to place one end of the veth into the container's network namespace (serving as the container's eth0) while keeping the other end on the host side. Packets sent from inside the container can then travel through this "virtual cable" to the host, ready to be connected to a bridge.

In the kernel, veth pairs are implemented in drivers/net/veth.c as a pair of mutually bound virtual devices. Data transfer happens entirely in kernel memory, with no physical hardware involved. After the sending side calls dev_queue_xmit() to enqueue a packet, the receiving side directly pulls it from the peer via the NAPI polling mechanism. The entire process involves no DMA or hardware interrupts, so latency is extremely low, and the throughput ceiling is typically constrained by CPU processing power rather than bandwidth. This also explains why network performance between containers on the same host is far higher than for cross-host communication.

Extended Background: Performance Tuning for veth pairs In high-throughput scenarios, the performance bottleneck of a veth pair is usually not bandwidth but the CPU's soft-interrupt processing overhead. Modern kernels optimize veth specifically through Generic Segmentation Offload (GSO) and Generic Receive Offload (GRO), allowing the kernel to defer segmentation when sending large blocks of data, reducing the number of system calls. Furthermore, the introduction of XDP (eXpress Data Path) allows packets to be processed at the earliest stage of the kernel network stack, bypassing most protocol stack overhead—next-generation CNI plugins like Cilium leverage exactly this feature to compress the CPU overhead of the forwarding path to a fraction of that of traditional iptables solutions, while retaining the veth topology.

Linux Bridge

If the veth pair solves the "point-to-point" connection problem, the bridge is responsible for "multipoint interconnection." A Linux Bridge is essentially a virtual switch operating at Layer 2, capable of attaching multiple network interfaces to the same broadcast domain so that connected devices can communicate with each other.

The Linux Bridge is implemented in the kernel by the bridge module, and its working mechanism closely resembles that of a physical switch: it maintains a MAC address learning table (FDB, Forwarding Database), learns the location of unknown destination MACs through flooding, and then forwards to known destinations according to the table. In a Docker environment, since the network topology is simple and there's no risk of loops, STP (Spanning Tree Protocol) is usually disabled (which you can confirm with brctl showstp br0) to avoid the problem of STP convergence delay causing brief communication failures right after container startup. In addition, the bridge itself can be assigned an IP address, in which case it acts as both a Layer 2 switch and a Layer 3 router—this is precisely the technical basis for it serving as the container's default gateway.

Docker's default docker0 is exactly such a bridge. Every time a container starts, Docker creates a veth pair and attaches the host-side end to docker0. This is the fundamental reason why multiple containers on the same host and the same network can ping each other.

Hands-On Experiment: Manually Reproducing Container Networking Step by Step

After the theory, the best way to learn is to reproduce it yourself. The following steps precisely reproduce all the network configuration work that Docker automatically performs behind the scenes.

Step 1: Create the Bridge and Network Namespace

Create a Linux bridge on the host and enable it:

ip link add br0 type bridge
ip link set br0 up

Create a network namespace to simulate a container:

ip netns add container1

At this point, container1 has a completely isolated network environment, but it's still an "island"—unable to communicate with the outside world.

Step 2: Connect the Namespace and Bridge with a veth pair

Create a veth pair, placing one end into the namespace and connecting the other to the bridge:

ip link add veth-host type veth peer name veth-c1
ip link set veth-c1 netns container1
ip link set veth-host master br0
ip link set veth-host up

Enter the namespace, configure an IP for the internal interface, and enable it:

ip netns exec container1 ip addr add 172.18.0.2/24 dev veth-c1
ip netns exec container1 ip link set veth-c1 up
ip netns exec container1 ip link set lo up

After completing this step, the "container" is connected to the bridge via veth, achieving Layer 2 interconnection with other namespaces attached to the same bridge.

Step 3: Enable External Network Access (Gateway + NAT)

Container access to the external network requires two more key configurations. First, assign an IP to the bridge and set it as the container's default gateway:

ip addr add 172.18.0.1/24 dev br0
ip netns exec container1 ip route add default via 172.18.0.1

Next, configure the iptables MASQUERADE rule so that packets sent from the container's private subnet are automatically translated to the host's IP when leaving the host:

iptables -t nat -A POSTROUTING -s 172.18.0.0/24 ! -o br0 -j MASQUERADE

MASQUERADE is a special form of SNAT (Source NAT); both reside in the POSTROUTING chain of the iptables nat table. Unlike SNAT, which requires explicitly specifying a fixed source IP, MASQUERADE dynamically obtains the current IP of the outbound interface as the translated source address, making it suitable for scenarios where the host IP may change. The cost is that it must query the outbound interface's IP each time a new connection is established, so in extremely high-concurrency scenarios you might consider switching to SNAT for a slight performance gain.

Extended Background: How conntrack Connection Tracking and NAT Work The NAT functionality of iptables relies on netfilter's connection tracking module (conntrack). When the first packet triggers a MASQUERADE rule, conntrack establishes a five-tuple record (source IP, source port, destination IP, destination port, protocol) in the kernel. Subsequent packets belonging to the same connection are translated by direct table lookup, without needing to traverse the rule chain again. This is the key mechanism that allows container networking to maintain stateful TCP connections. However, in large-scale cluster scenarios, the conntrack table's capacity limit (controlled by the nf_conntrack_max parameter, typically 65536 by default) and hash table lock contention become significant performance bottlenecks—this is also one of the core motivations behind the Kubernetes community's push to migrate from iptables to eBPF/nftables. When troubleshooting NAT anomalies, you can use conntrack -L to view the current tracking table state, and conntrack -S to check the insert_failed count in the statistics to determine whether packet loss is occurring due to a full table.

Finally, enable kernel IP forwarding to allow packets to flow between different network interfaces:

echo 1 > /proc/sys/net/ipv4/ip_forward

At this point, a fully functional "container network" capable of accessing the external network has been built by hand—and this is precisely all the work that Docker's default bridge mode does behind the scenes.

Compared with Docker: Mapping Manual Steps to Automation

Looking back and comparing the manual steps, Docker's automation logic becomes clear: when a container starts, it automatically creates the docker0 bridge, assigns each container an independent network namespace and veth pair, manages IP address allocation through IPAM, and automatically writes the corresponding iptables NAT rules. The essence of port mapping (-p 8080:80) is an iptables DNAT rule that forwards inbound traffic on a specified host port to a port inside the container.

Understanding this mapping resolves many common problems:

• The container can access the internet, but the internet can't actively access the container? MASQUERADE only handles outbound traffic; inbound access requires an explicit port mapping (DNAT rule) to take effect.
• Why can containers on the same host communicate by default? Because they are attached to the same docker0 bridge, in the same broadcast domain, directly reachable at Layer 2.
• How to troubleshoot port mapping failures? Prioritize checking whether the DNAT rule exists in the iptables NAT table and whether IP forwarding (ip_forward) is enabled—this usually quickly pinpoints the root cause.

Lay a Solid Foundation for Efficient Future Learning

Container networking may seem complex, but it's actually an elegant combination of a few "building blocks": the network namespace, the veth pair, the Linux Bridge, and iptables. When you can reproduce Docker's default network by hand, it means you've truly grasped the first principles of container networking.

The value of this foundational understanding continues to compound as you dive deeper into the tech stack. CNI (Container Network Interface) is a container networking standard interface specification defined by CNCF, which requires network plugins to implement three operations: ADD, DEL, and CHECK. Flannel's host-gw mode essentially manipulates the routing table on each node, pointing cross-node Pod subnets to the corresponding host IP, still relying on veth pairs and bridges underneath. Its VXLAN mode builds on this by encapsulating Layer 2 frames within UDP packets via VTEP devices, implementing an overlay network across Layer 2 domains. Calico, on the other hand, abandons the bridge entirely, distributing routes via the BGP protocol and directly using the host end of the veth as the routing next hop—achieving higher forwarding performance while maintaining pure Layer 3 routing.

Extended Background: The Technical Divergence and Selection Logic of CNI Plugins The CNI specification itself is extremely minimal; its design deliberately avoids constraining the underlying implementation, requiring only that plugins exist as executable files, receive JSON configuration via standard input, and obtain the container namespace path via environment variables. This "minimal interface" design has led to a significant technical divergence in the CNI ecosystem: overlay solutions represented by Flannel prioritize network topology compatibility, sacrificing some performance for the ability to deploy across heterogeneous networks; underlay/BGP solutions represented by Calico pursue maximum forwarding efficiency, given controllable network infrastructure; while eBPF solutions represented by Cilium attempt to fundamentally replace the netfilter path, compressing forwarding latency to near kernel-bypass levels while retaining flexible policy capabilities. Choosing a CNI plugin is essentially a trade-off decision across three dimensions: operational complexity, network performance, and infrastructure constraints. And regardless of the solution, its implementation approach is built upon the very set of network primitives you just set up manually.

No matter which CNI plugin you use, its implementation approach is a further abstraction and extension built upon these fundamental primitives: the network namespace, the veth pair, and routing/iptables. With a solid foundation, advanced learning naturally falls into place.

Key Takeaways

• The network namespace provides thorough network stack isolation and is the foundation of containerization; the seven Linux Namespace types are mutually orthogonal and can be flexibly combined.
• The veth pair is the "virtual network cable" connecting a namespace to the external network; transfers happen in kernel memory with extremely low latency.
• The Linux Bridge acts as a Layer 2 virtual switch, maintaining an FDB table to interconnect multiple containers, while also serving as a Layer 3 gateway.
• iptables MASQUERADE + conntrack is key to container internet access; the conntrack table capacity is a potential bottleneck in large-scale scenarios.
• Docker's automation is essentially a programmatic encapsulation of the above manual steps, and port mapping is a DNAT rule; understanding the manual steps is the shortest path to troubleshooting.
• CNI plugins, no matter how they evolve, are all built upon this set of fundamental primitives; mastering the underlying layer makes selection and troubleshooting much easier.