feat(bpf): CO-RE socket-birth eBPF program + skip-safe skeleton build#62
Open
ebenali wants to merge 4 commits into
Open
feat(bpf): CO-RE socket-birth eBPF program + skip-safe skeleton build#62ebenali wants to merge 4 commits into
ebenali wants to merge 4 commits into
Conversation
The production sibling of the bench's validated birth.bt: a CO-RE eBPF
program that captures a flow's owning pid + direction + 5-tuple at BIRTH
(connect/accept/first connected UDP send), in the owner's context before a
short-lived process can exit — the gap conntrack+sock_diag misses ~100% of
the time under churn.
Program (src/backend/linux/bpf/):
- birth.bpf.c — four BPF-trampoline probes mirroring birth.bt:
fexit/tcp_v{4,6}_connect (outbound; at RETURN the ephemeral port is
assigned), fentry/udp_sendmsg (connected UDP, per-sock LRU dedup),
fexit/inet_csk_accept (inbound; return is the child sock). Births → a
ring buffer. fexit/fentry over k(ret)probes: reads args AND return with
no pt_regs/arch register macros and no entry→return stash map, and is
cheaper. Needs CONFIG_FUNCTION_TRACER+BTF (standard on distro kernels
6.6–7.1); skip-safe to conntrack-only otherwise.
- vmlinux_min.h — minimal CO-RE type subset (preserve_access_index) so
the BUILD host needs no kernel BTF; CO-RE relocates field offsets at
load. No multi-megabyte generated vmlinux.h.
- birth_events.h — byte-for-byte wire struct shared with the userspace
ring-buffer reader (lands next PR).
Build (gated on QIFTOP_HAVE_BPF):
- src/backend/linux/CMakeLists.txt: clang -target bpf → birth.bpf.o →
bpftool gen skeleton → birth.skel.h (embeds the object, so nothing
extra ships). Arch-derived __TARGET_ARCH. backend_linux depends on the
skeleton so CI compiles + validates the program on every push.
- CI installs the toolchain best-effort (clang/libbpf-dev/bpftool;
reliable on fedora, best-effort on ubuntu) — skip-safe where absent.
- vagrant provisioner gains clang/libbpf-dev/bpftool for VM load tests.
Docs: AGENTS.md §8b documents the program, the fexit-vs-kprobe rationale,
the minimal-CO-RE/no-BTF-at-build decision, and the embedded-skeleton
build. Validated locally: compiles to a valid eBPF object, skeleton
generates (4 progs / 2 maps), full build + 43/43 tests + clang-tidy gate
clean. Accretes toward v0.4.0 / contract 0.7.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ebenali
enabled auto-merge (squash)
June 19, 2026 19:52
…uard /proc/<pid>/stat field 22 (what the userspace starttime probe reads) is derived from task->start_boottime via nsec_to_clock_t(timens_add_boottime_ns(start_boottime)) — NOT start_time. Capturing start_time would make the BpfBirthResolver PID-reuse guard mismatch every time (the captured ticks could never equal the probed field-22 ticks), so a long-lived process's birth would be false-rejected. Read start_boottime via CO-RE and rename the wire field start_boottime_ns to match. Verified: recompiles + skeleton regenerates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…+ gone-pid guard fix (#63) * feat(bpf): userspace ring-buffer reader + birth decoder + gone-pid guard fix Wires the kernel side of the birth+conntrack hybrid to userspace: - backend/linux/BpfBirthReader.{h,cpp} (gated on QIFTOP_HAVE_BPF): loads the CO-RE skeleton (birth.skel.h), attaches the fexit/fentry probes, and drains the ring buffer on a dedicated std::thread, feeding each birth to a Sink callback. Skip-safe — start() returns false and stays inert on any kernel without BTF / trampolines / CAP_BPF, so the chain runs conntrack-only. Routes libbpf logging into the verbose category. - backend/linux/BirthDecode.h: the PURE wire-event → BirthKey/BirthRecord decoder (no kernel/libbpf, so it's always built + tested). Network-order bytes → QHostAddress (v4/v6), host-order ports, IANA proto → L4Proto, direction byte → Direction, and start_boottime_ns → /proc field-22 clock ticks via sysconf(_SC_CLK_TCK). test_birth_decode pins the byte-order/units contract with hand-built events. - CMake: backend_linux gains BpfBirthReader.cpp + links libbpf (gated). Correctness fix to the merged BpfBirthResolver PID-reuse guard: it rejected when live==0 (pid gone), which would reject exactly the short-lived processes the hybrid exists to capture (they've exited by the time the conntrack flow resolves). Now it serves a gone pid and rejects ONLY when a DIFFERENT live process holds the pid (live!=0 && live!=captured). Test flipped accordingly (missingPidStarttimeIsRejected → goneShortLivedPidIsServed). Also fixes the captured starttime field (start_time → start_boottime, the one /proc field 22 derives from) so the guard's equality check is sound. AGENTS.md §8b documents the reader, the decoder units contract, and the gone-pid guard semantics. Validated: full build + 44/44 tests + clang-tidy gate clean (HAVE_BPF on, toolchain present); skip-safe off-build verified. Factory wiring (construct resolver first, point reader at it, advertise the birth-attribution token) is the next PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(bpf): per-probe tolerant attach in the birth reader Attach each eBPF program individually (bpf_program__attach over bpf_object__for_each_program) and keep the ones that take, instead of the skeleton's all-or-nothing qiftop_birth__attach. A kernel where one traced function is renamed/inlined/non-attachable (or hits the trampoline limit) still yields birth attribution from the probes that DID attach; start() succeeds when >=1 probe attaches and logs "attached N/M probes". We own the returned bpf_links and destroy them in stop(). Load stays whole-object on purpose: CO-RE relocation + verification is per-object, so if load fails nothing is attachable anyway. Still fully skip-safe — zero probes attached tears down and returns false (conntrack-only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
) * feat(bpf): userspace ring-buffer reader + birth decoder + gone-pid guard fix Wires the kernel side of the birth+conntrack hybrid to userspace: - backend/linux/BpfBirthReader.{h,cpp} (gated on QIFTOP_HAVE_BPF): loads the CO-RE skeleton (birth.skel.h), attaches the fexit/fentry probes, and drains the ring buffer on a dedicated std::thread, feeding each birth to a Sink callback. Skip-safe — start() returns false and stays inert on any kernel without BTF / trampolines / CAP_BPF, so the chain runs conntrack-only. Routes libbpf logging into the verbose category. - backend/linux/BirthDecode.h: the PURE wire-event → BirthKey/BirthRecord decoder (no kernel/libbpf, so it's always built + tested). Network-order bytes → QHostAddress (v4/v6), host-order ports, IANA proto → L4Proto, direction byte → Direction, and start_boottime_ns → /proc field-22 clock ticks via sysconf(_SC_CLK_TCK). test_birth_decode pins the byte-order/units contract with hand-built events. - CMake: backend_linux gains BpfBirthReader.cpp + links libbpf (gated). Correctness fix to the merged BpfBirthResolver PID-reuse guard: it rejected when live==0 (pid gone), which would reject exactly the short-lived processes the hybrid exists to capture (they've exited by the time the conntrack flow resolves). Now it serves a gone pid and rejects ONLY when a DIFFERENT live process holds the pid (live!=0 && live!=captured). Test flipped accordingly (missingPidStarttimeIsRejected → goneShortLivedPidIsServed). Also fixes the captured starttime field (start_time → start_boottime, the one /proc field 22 derives from) so the guard's equality check is sound. AGENTS.md §8b documents the reader, the decoder units contract, and the gone-pid guard semantics. Validated: full build + 44/44 tests + clang-tidy gate clean (HAVE_BPF on, toolchain present); skip-safe off-build verified. Factory wiring (construct resolver first, point reader at it, advertise the birth-attribution token) is the next PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(bpf): per-probe tolerant attach in the birth reader Attach each eBPF program individually (bpf_program__attach over bpf_object__for_each_program) and keep the ones that take, instead of the skeleton's all-or-nothing qiftop_birth__attach. A kernel where one traced function is renamed/inlined/non-attachable (or hits the trampoline limit) still yields birth attribution from the probes that DID attach; start() succeeds when >=1 probe attaches and logs "attached N/M probes". We own the returned bpf_links and destroy them in stop(). Load stays whole-object on purpose: CO-RE relocation + verification is per-object, so if load fails nothing is attachable anyway. Still fully skip-safe — zero probes attached tears down and returns false (conntrack-only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(bpf): wire eBPF birth into the resolver chain (BpfBirthSource) Adds the factory integration that lights up birth attribution end-to-end: - backend/linux/BpfBirthSource: a ProcessResolver adapter that owns the BpfBirthReader AND a BpfBirthResolver, feeds the reader's births into the resolver, and gates setLoaded(true) (hence its capability advertisement) on a SUCCESSFUL program attach. Skip-safe: initialize() returns false when the program can't load/attach (no BTF / trampolines / CAP_BPF) and the factory simply doesn't add it. Member order ensures the reader thread is joined before the resolver it feeds is torn down. - ProcessResolverFactory: construct BpfBirthSource FIRST in the composite (under BACKEND_LINUX && QIFTOP_HAVE_BPF, gated on cfg.processAttribution), so birth wins for flows it saw and sock_diag attributes the rest. Because both the agent (InterfacesService merges resolver caps) and the in-process ConntrackMonitor build via this one factory, the birth-attribution + process-attribution[-wire] tokens flow to the wire automatically — no service changes, no usingAgent assumption. - BpfBirthResolver: periodic TTL prune (every 4096 inserts) so unmatched births don't pile to the hard cap and trigger clear-on-overflow that would drop FRESH births on churny hosts. find() only TTL-filters (it's const), so this keeps the live set well below the cap as a routine matter. Full build + 44/44 tests + clang-tidy gate clean (HAVE_BPF on). Skip-safe off-build unaffected. Agent-level VM validation follows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The production sibling of the bench's validated
birth.bt: a CO-RE eBPF program that captures a flow's owning pid + direction + 5-tuple at BIRTH (connect/accept/first connected UDP send), synchronously in the owner's context before a short-lived process can exit — the gap conntrack+sock_diag misses ~100% of the time under churn (Phase-0 finding). conntrack still supplies the bytes; birth supplies the attribution.Program —
src/backend/linux/bpf/birth.bpf.c— four BPF-trampoline probes mirroringbirth.bt:fexit/tcp_v{4,6}_connect(outbound; at RETURN the ephemeral port is assigned — the SYN_SENT tracepoint fires too early,local_port=0),fentry/udp_sendmsg(connected UDP, per-sock LRU dedup),fexit/inet_csk_accept(inbound; return is the child sock). Births →BPF_MAP_TYPE_RINGBUF.vmlinux_min.h— minimal CO-RE type subset underpreserve_access_index, so the BUILD host needs no kernel BTF (CO-RE relocates offsets at load); no multi-MB generatedvmlinux.h.birth_events.h— byte-for-byte wire struct shared with the userspace ring-buffer reader (next PR).Why fexit/fentry, not the bench's k(ret)probes
Reads args and return value with no
pt_regs/arch-register macros and no entry→return sock-stash map, and is cheaper. Semantically identical (fexit-on-connect == the validated kretprobe). Tradeoff: needsCONFIG_FUNCTION_TRACER+ BTF — standard on distro kernels 6.6→7.1; skip-safe to conntrack-only otherwise.Portability (asked)
CO-RE relocates every field access against the target kernel's BTF, so one object loads unchanged across 6.6→7.1; the four traced functions have stable signatures across that range. Kernels without trampoline/BTF support fall back cleanly.
Build (gated on
QIFTOP_HAVE_BPF)clang -target bpf→birth.bpf.o→bpftool gen skeleton→birth.skel.hwhich embeds the object (nothing extra ships in the package; the program rides in the agent binary).backend_linuxdepends on the skeleton, so CI compiles + validates the program on every push wherever the toolchain is present. CI installs the toolchain best-effort (reliable on Fedora; best-effort on Ubuntu) — skip-safe where absent. Vagrant provisioner gains the toolchain for VM load tests.Verification
Validated locally (Debian clang 22, libbpf 1.7, bpftool 7.7): compiles to a valid eBPF object, skeleton generates (4 progs / 2 maps, CO-RE relocations present), full build + 43/43 tests + clang-tidy gate clean.
birth.bpf.ccorrectly excluded from the C++ clang-tidy set. Load-testing on a real BTF kernel is the follow-up (the userspace reader PR + Tier-2 VM test).Accretes toward v0.4.0 / contract 0.7 (no version bump, no new token here — the
birth-attributiontoken ships with the resolver-wire PR).