eBPF Explained: Extending the Kernel Without Recompiling It

Cilium, Falco, Pixie, Katran. For a few years now the word eBPF has appeared on the landing page of every other infrastructure tool. The conclusion up front: eBPF is a technology for running small, verified programs safely inside the running kernel — no recompilation, no kernel modules. Call it “making the kernel programmable”; observability, networking, and security tooling have all migrated onto it over the past decade. This post covers the mechanism and the landscape.

The problem: seeing what happens inside the kernel #

The truth about a system lives in the kernel — which process opens which file, where packets get dropped, where disk I/O stalls. But the traditional ways of looking inside or changing behavior both had problems.

• Kernel modules: can do anything, but one bug is a kernel panic. Risking the whole server on one module is a hard sell in production.
• Userspace tools: safe but slow and limited. Copying every packet up to userspace for inspection, for instance, makes the copying itself the bottleneck.

eBPF is the third road: run small programs in an in-kernel virtual machine, but have a verifier statically prove their safety before loading. You get the kernel’s speed and vantage point without carrying the module’s risk.

How it works: the verifier as gatekeeper #

An eBPF program’s life cycle:

1. The program (usually written in C, compiled to bytecode) is loaded into the kernel.
2. The verifier inspects every execution path. Loops without guaranteed termination, memory access outside permitted bounds, use of uninitialized values — any of these and the load itself is rejected. This gatekeeper is why something running inside the kernel cannot kill the kernel.
3. Programs that pass are JIT-compiled and run at native speed.
4. The program attaches to a hook — called every time a specific kernel event fires.
5. Collected data accumulates in maps, in-kernel data structures that userspace tools read out.

The hook points map directly to the use cases.

What it is used for: three branches #

Observability is the most approachable branch. The old ways required instrumenting the application or attaching sampling profilers; eBPF pulls data straight from kernel events, so you see the whole system without modifying or restarting the target processes. Questions like “which process is opening which files,” “where are TCP retransmits coming from,” and “what does the disk-latency distribution look like” resolve into one-liners. It is the next layer down when the standard tools of slow-API diagnosis run out of answers.

Networking is the branch where eBPF changed the industry. XDP runs eBPF programs at the NIC driver level, before packets even enter the kernel network stack — millions of packets per second of filtering and forwarding, which is what DDoS defenses and L4 load balancers are now built on. In Kubernetes, Cilium implements pod networking, policy, and load balancing in eBPF instead of iptables chains, becoming a de facto standard axis among CNIs.

Security watches syscalls and kernel events in real time to detect and block policy violations. Runtime security tools like Falco and Tetragon catch “this container suddenly spawned a shell” and “that wrote to an unexpected path” on an eBPF foundation.

Trying it yourself: the tool ladder #

You do not need C and kernel internals to use eBPF. There is a ladder.

1. bpftrace: an awk-like one-liner language — the entry point with the best effort-to-payoff ratio.
2. The bcc tool collection: dozens of prebuilt tools install as a package — execsnoop (trace process launches), opensnoop (file opens), biolatency (disk-latency histograms), tcpretrans (TCP retransmits). Most operational questions end with these off-the-shelf tools.
3. libbpf + CO-RE: the stage for building your own tools and products. CO-RE (Compile Once, Run Everywhere) removed most of the old pain of recompiling per kernel version.

# bpftrace one-liner: files being opened on the system, live
bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)); }'

Practical constraints #

• Privileges: loading eBPF requires root (or fine-grained capabilities like CAP_BPF) by default. If any process could instrument the kernel, that would itself be the security hole.
• Kernel versions: features are tightly bound to kernel versions. The newer the feature, the newer the kernel required — on older distributions, checking the tool’s required kernel version is step one.
• Managed environments: eBPF from inside containers tangles with host-kernel privileges, and some managed services restrict it. That is why node-daemon (DaemonSet) deployment is the standard pattern.
• Not a universal tool: programs operate within the verifier’s constraints (size, complexity), so complex logic is still designed as a collaboration with userspace.

Summary #

• eBPF runs small verified programs safely inside the kernel — the kernel’s vantage point and speed without the module’s risk.
• The heart of the safety is the verifier: programs without provable termination or with unsafe memory access are rejected at load time.
• The uses fall into three branches: observability (no-modification, no-restart instrumentation), networking (XDP, Cilium), and security (runtime detection like Falco).
• Start with bpftrace one-liners and the prebuilt bcc tools; drop down to libbpf + CO-RE when building products.
• Root privileges and kernel-version dependence are the practical constraints — evaluation starts with “does this tool run on our kernel?”