test - #1802
Draft
qmonnet wants to merge 16 commits into
Draft
Conversation
qmonnet
added
ci:+release
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
The kernel driver reads packets with AF_PACKET, which copies every frame through a socket buffer and caps what one worker can move. AF_XDP lets the kernel put received packets straight into memory we share with it, and take transmitted ones from the same place, but nothing in the tree describes that memory or presents it to the pipeline. The crate starts with the part that has no dependency on libxdp: how a frame is laid out, and a packet buffer that reads and writes one in place. Because prepending and trimming only move offsets within the frame, a packet can cross the pipeline without being copied anywhere. The libxdp-dependent parts sit behind the "runtime" feature, so the default build, and the buffer's own tests, need neither libxdp nor the C toolchain it is built with. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The AF_XDP support in the dataplane-xdp crate binds sockets through libxdp, which the build compiles from source along with the libbpf it bundles. That build needs three things we do not have: libelf and zlib to link against, m4 to preprocess libxdp's dispatcher program, and a compiler that will accept "-target bpf" for the BPF programs libxdp carries. The last one is the awkward one. Our clang is wrapped, and the wrapper hands it hardening flags such as -fzero-call-used-regs, which clang rejects outright for the bpf target. The build honours CLANG for exactly those compilations, so point it at the compiler under the wrapper and leave everything else building the way it did. The dev shell and the packaged build each need all three, so both get them. Only the static libraries go in the sysroot, as for everything else there: the dataplane ships in an image with no shared libraries of its own to find, and one that wanted libelf.so at startup would not run. Static linking is what makes elfutils' own crc32 collide with zlib's, and what makes its optional compressors -- which are its business, not libbpf's -- pull three more libraries into the link, so both are dropped. elfutils is also built without debuginfod, which would want curl for a feature we have no use for, and with its test suite off: it does not survive our stdenv and covers nothing on the path we use. None of that is quite the same for a cross build, where the tools left in the environment under their plain names are the build host's. The ar that drops elfutils' crc32 is not there at all; libbpf's build looks for an unprefixed pkg-config and finds none; and the ld and objcopy libxdp wraps its BPF programs with hand back objects for the build host, which the link for the target then refuses. Each is pointed at the tool for the target instead. On aarch64 there is one more: gcc reaches into libgcc for its atomics, and nothing in the sysroot carries it, so libbpf is told to emit them inline. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
A worker needs a socket on every interface it serves, not just the one it receives from, or a packet could not be forwarded. Binding each of those to a UMEM of its own would give the worker one free-frame pool per interface, and leave a packet arriving on one interface unable to leave by another without a copy between mappings. So the UMEM comes first and the sockets are bound to it: XskUmem owns the mapping and the free list, and hands out an XskSocket per (interface, queue) pair, each with its own four rings. A frame is on exactly one ring, on the free list, or in a buffer somewhere in the pipeline, and finds its way back to the free list from wherever it ends up -- including from a buffer the pipeline drops, which returns its frame over a channel. Zero-copy is tried first and copy mode used if the driver refuses, so a NIC without zero-copy support runs slower rather than not at all. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
libxdp loads a redirect program when a socket is bound, which is enough to get packets to userspace but leaves the redirect decision in a program we do not own and cannot change. Anything we later want to do before a packet reaches the pipeline -- filtering, steering, counting -- has to live in that program. So we bring our own, and tell libxdp not to load one. The program itself is a few lines: look the RX queue up in a map of sockets, redirect if there is one, and pass to the kernel stack if there is not, so traffic on queues we do not serve is untouched. Around it are the pieces that get it built and loaded: a crate outside the workspace, because it is compiled for the BPF target with a nightly toolchain and bpf-linker; a just recipe to drive that build; and a build script that compiles the resulting object into the loader. Nothing runs the recipe yet, so for now the object is whatever whoever is building has left in the target directory, or points DATAPLANE_XDP_EBPF at. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The program in xdp-ebpf/ has to be compiled by hand, by whoever has a bpf-linker and a nightly toolchain, which is not a thing a build can depend on: nothing that needs the driver can be built without it. Three things stood in the way, and none of them needed a nightly toolchain in the end. RUSTC_BOOTSTRAP is enough for the -Zbuild-std the BPF target needs because it has no prebuilt std. bpf-linker is in nixpkgs, but built against the LLVM of the nixpkgs rustc, and it reads the bitcode ours emits -- so it is rebuilt against the LLVM our own rustc reports, which is what llvmPackages' already is. And the program is built by a derivation of its own rather than by the workspace, since it is a different target linked by a different linker; the result reaches the dataplane build through DATAPLANE_XDP_EBPF. The profile it is built with is not a matter of taste. The verifier refuses a program whose last instruction is a call, and a debug build puts the null and alignment checks for a raw pointer dereference after the program's exit; it would not load at all. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
Kif is how a driver names the interfaces it was told to serve: the ifindex to tag packets with, and the name to bind on. None of that is specific to AF_PACKET, and a second driver needs the same table built the same way, from the same command line, with the same interfaces brought up first. Along with the move it gains the RX queue count, which a driver that binds one socket per queue needs to decide how many workers to run and which queues to serve. Interfaces whose queue count sysfs will not report are treated as having one, which is what the kernel gives them anyway. The af-xdp feature is declared here so the queue count, which only that driver uses, is not compiled into a build that has no use for it. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
start_router was generic over the buffer type, which meant the buffer type of the driver decided the type of everything it returned -- the router handle, the writers management holds, the stats collector -- even though none of those depend on it. With one driver the compiler inferred it from the driver's own signature and nobody noticed. With two, whose buffers differ, the same call cannot serve both. Only building a pipeline actually needs to know the buffer type, so that is where the parameter goes: start_router hands back a factory holding what a pipeline is assembled from, and the driver asks it for a pipeline over the buffer it reads into. The stages themselves already work for any buffer. The reader factories the pipeline is built from were public types in private modules, nameable only as the return type of a getter, so a struct could not hold them. Re-export the three that were not already. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The supervisor watches worker threads: it joins the ones that end, calls stuck the ones whose rx tasks stop patting their watchdogs, and publishes what they all report. None of that depends on how a worker gets its packets, but it was written inside the kernel driver, where a second driver cannot reach it. Move it, along with the monitors it watches through, and give it the periods it works to rather than reading them off one driver. The status it publishes carried the same assumption -- it printed the kernel driver's batch size and periods whatever driver was running -- so those become part of what a driver reports about itself, next to its name. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The kernel driver moves packets with AF_PACKET, which copies every frame through a socket buffer on the way in and again on the way out. AF_XDP does away with both copies: the kernel writes received packets into memory the worker already has mapped, and reads transmitted ones from the same place. An AF_XDP socket is bound to one RX queue of one interface, so the workers are organised by queue rather than by fanout: worker q owns a socket on queue q of every interface, and the queue count decides how many workers there are. Behind those sockets is one UMEM per worker, so a packet arriving on one interface can leave by another without being copied between mappings, and the worker draws frames from a single pool rather than one per interface. Where the kernel driver waits on tokio, this one blocks in poll(2) across its sockets. That poll's timeout is also what bounds how long the worker takes to notice it has been cancelled, so there is no async machinery in the loop at all. The driver is behind the af-xdp feature, since libxdp has to be built from source and needs a C toolchain the default build does without. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
libxdp pins the redirect program it loads under /sys/fs/bpf, and uses those pins to count the sockets on an interface so it knows when the last one has gone and the program can be detached. Without the filesystem it says so, falls back to a program it cannot later detach, and every interface the dataplane ran on keeps one after it exits -- which the next run then has to displace. Most systems have the filesystem mounted. A container often does not, and a container is where the dataplane runs, so mount it rather than making whoever launches us do it. Failing to is not fatal: it costs the clean detach, not the ability to move packets, so it is reported and the driver carries on. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The AF_XDP driver asks no more of a NIC than the kernel driver does, and moves packets without the copy through a socket buffer that the kernel driver pays for on every frame. There is no reason to prefer the kernel driver where both will run, and every reason for what CI and the VLAB exercise to be what we ship, so it becomes what the dataplane runs when it is not told otherwise. Being the default means being built in, so the af-xdp feature joins the default set. That makes libxdp, and the XDP program built for the BPF target, part of an ordinary build; a build without the toolchain for them can still turn the feature off, and then has to be told to use the kernel driver, which it says plainly rather than reporting the driver it defaults to as unknown. --driver kernel still selects the kernel driver, for anywhere the process cannot be given the capabilities AF_XDP needs. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The dataplane has had a --driver flag with more than one answer for a while, and now has a third, which it uses unless told otherwise. Nothing said what they are, which one runs by default, or what that one asks of the system it runs on. Say so. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
A packet the pipeline marks Local is one it wants the host stack to have. No driver delivers it: nothing in userspace can inject into an interface's receive path, so both of them drop it. The kernel driver gets away with that, because AF_PACKET leaves the kernel its own copy of every frame regardless of what we do with ours. The AF_XDP driver does not: an XDP program that redirects a packet to a socket is the last thing to see it. So under AF_XDP these are packets the host was supposed to receive and did not, and they were leaving no trace at all -- not counted as pipeline drops, since the pipeline handed them back, and not as TX drops, since no send was attempted. Give them a counter of their own, reported next to the drops, so the gap is a number rather than a mystery. Both drivers set it, so it means the same thing whichever is running, even though only one of them is losing anything. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
An XDP program that redirects a packet to an AF_XDP socket is the last thing to see it: the network stack never does. libxdp's program redirects everything on a queue with a socket, so with the driver running, anything on those interfaces that the host itself was expecting -- routing sessions, neighbour discovery, anyone pinging the gateway -- stopped arriving. Nothing in userspace can put those packets back: an AF_PACKET write transmits, and only a tap can be written into, which is not the interface the packet came in on. The decision has to be made before the redirect, which means in the program. So the program classifies. What is not IP goes to the kernel, which is what the pipeline does with it anyway. What is addressed to one of the gateway's own addresses goes to the kernel, which is what the pipeline marks Local for. VXLAN is the exception on both counts: it arrives addressed to the gateway and it is exactly what the dataplane exists to handle. Everything else, which is the traffic being forwarded, is redirected as before. Which addresses are the gateway's own is not known at startup -- they arrive with its configuration and change with it -- so the driver reads them back from the kernel and hands them to the program repeatedly. Polling rather than following netlink is deliberate: the set is rebuilt from what the kernel holds every time, so a missed message cannot leave the program permanently wrong, and the cost is a bounded delay before a new address stops being redirected. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
The kernel refuses to register a UMEM whose chunks are larger than a page, so a frame holds about 3.8KB of packet and no more. That is well under the 9100 the fabric runs at, and the consequences were not subtle: the kernel will not attach an XDP program that has not said it copes with fragments to an interface whose MTU exceeds a buffer, so ours only ever attached in generic mode, and anything too long to fit was dropped without reaching the pipeline. Measured on a veth at MTU 9100, a 1.4KB ping arrived and an 8KB one did not. Multi-buffer is the kernel's answer: a long packet arrives as several descriptors in a row, each but the last saying there is more to come, and leaves the same way. The socket asks for it at bind, and the program declares it so that a native attach is allowed again. The pipeline needs a packet as one run of bytes -- it parses and rewrites headers, and prepends to them -- and the fragments are in frames nowhere near each other, so a packet that arrives in pieces is gathered into a buffer of its own and split again on the way out. That is a copy each way, paid only by packets too long for a frame; a packet that fits one still crosses the pipeline without being copied at all. Doing better would mean teaching PacketBufferMut about scatter-gather, which is a change to every buffer in the tree rather than to this driver. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
Making AF_XDP the default did not change what the VLAB or CI run. The gateway controller in the fabric repo decides the driver from whether a gateway's interfaces are named by PCI address or by kernel name, and for the latter it appends --driver kernel whatever else is true. An explicit flag wins over the default, as it should, so nothing here can reach that deployment: asking for AF_XDP there means a change in fabric and another in fabricator, whose list of gateway drivers has two entries in it. Until those land, serve that name from the AF_XDP driver. It is a stopgap and reads like one -- one arm, one warning saying what happened, and the AF_PACKET driver still reachable by its own name, which is what it should have been called all along. Keeping the name is also what makes the rest of the deployment work: the same controller only runs the init container that sets the interface MTU, turns GRO off and brings the links up when it thinks the driver is "kernel". Asking it for "af-xdp" would have skipped all of that, and the MTU it sets is the one the multi-buffer work exists to carry. A build without the AF_XDP driver still reads --driver kernel as the AF_PACKET one, since there it cannot mean anything else. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>
qmonnet
force-pushed
the
test/qmonnet/afxdp-driver
branch
from
September 5, 2026 00:13
d66dbc2 to
1bbcc33
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.