Skip to content

(09) pipeline - #1761

Draft
daniel-noland wants to merge 59 commits into
pr/daniel-noland/icmp-and-aclfrom
pr/daniel-noland/pipeline-harness
Draft

(09) pipeline#1761
daniel-noland wants to merge 59 commits into
pr/daniel-noland/icmp-and-aclfrom
pr/daniel-noland/pipeline-harness

Conversation

@daniel-noland

@daniel-noland daniel-noland commented Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

No description provided.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.24895% with 64 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
config/src/external/overlay/algebra.rs 93.63% 30 Missing and 20 partials ⚠️
routing/src/testing.rs 85.86% 13 Missing ⚠️
config/src/external/overlay/vpcpeering.rs 98.14% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 73e17bd to 03c87f1 Compare August 26, 2026 17:30
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch from 8cb37c3 to 8027a81 Compare August 26, 2026 17:30
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 03c87f1 to d200140 Compare August 26, 2026 19:36
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch 2 times, most recently from cf9c73e to fdb5ee2 Compare August 26, 2026 20:41
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from d200140 to e431baa Compare August 26, 2026 20:41
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch from fdb5ee2 to 70987f0 Compare August 26, 2026 21:02
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 30ad436 to 972ede1 Compare August 26, 2026 21:13
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch 2 times, most recently from 033504b to f2eee32 Compare August 26, 2026 21:25
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch 2 times, most recently from 86a422e to 40180ec Compare August 27, 2026 01:29
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch from f2eee32 to d1f4f0d Compare August 27, 2026 01:29
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 40180ec to ebe87d1 Compare August 27, 2026 01:41
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch 2 times, most recently from b712bef to 66a7cd4 Compare August 27, 2026 02:25
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 5d8566c to ee0870c Compare August 27, 2026 04:36
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch from 66a7cd4 to 67eba5b Compare August 27, 2026 04:37
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from ee0870c to d39b0d7 Compare August 27, 2026 05:12
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch 2 times, most recently from 0294067 to 11b3757 Compare August 27, 2026 06:30
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 543ccfc to 792d63f Compare August 27, 2026 18:00
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch from 11b3757 to 7346f90 Compare August 27, 2026 18:00
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 792d63f to 6c9c867 Compare August 27, 2026 18:29
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch from 7346f90 to fac79e0 Compare August 27, 2026 18:29
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/icmp-and-acl branch from 6c9c867 to d4744be Compare August 28, 2026 02:13
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/pipeline-harness branch 2 times, most recently from 1f1d323 to 86c8dca Compare August 28, 2026 02:19
daniel-noland and others added 30 commits September 17, 2026 13:43
@daniel-noland @claude
feat(routing): expose forwarding-table fixtures for tests
be3f5e0 
The pipeline harness could construct routing readers but had no supported
way to populate the tables behind them, blocking underlay forwarding tests.
Add a RouterTables test builder for VRFs, routes, next hops, interfaces, and
adjacencies, and expose only the readers consumed by production stages.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): verify routed packet delivery
96cc612 
The VLAN rejection in IpForwarder had no direct test, and an end-to-end drop
could still pass after removing it because downstream filters reject the
same packet. Add checks at both the decapsulation boundary and the full
routed pipeline, plus a deliverable positive control that proves the harness
can reach the wire.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
fix(fuzz): let pipeline generators use complete inputs
eaa2af2 
Bolero silently zero-filled data after its 4096-byte limit, making large
batches end in default values. LibFuzzer also kept these slow targets at
eight-byte inputs, so raising the generator limit alone changed nothing.
Align the input budgets, allow immediate length growth, and assert that each
generator fits the configured maximum.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
feat(fuzz): run campaigns across half the host
587876f 
A single fuzz worker left most CPU capacity idle. EAL targets already isolate
each process, so memory is the limiting resource rather than shared state.
Run independent workers on half the available cores, sized from the measured
1.8 GB footprint per pipeline worker.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
fix(fuzz): isolate coverage corpora from unit tests
dd65789 
Fuzz campaigns stored their coverage corpus where normal tests replayed it
under the same budget as random generation. A growing corpus then crowded
out fresh cases and caused sound coverage guards to fail. Keep campaign
inputs outside the source replay directory while retaining the small crash
corpus that unit tests should reproduce.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): round-trip replies through VXLAN
8e16cc1 
Reply-side decapsulation was untested because earlier overlay tests stamped
packet arrival by hand. Build the reply as a real tunnelled frame under the
peer VNI and follow it through forwarding and reverse translation. Set a
usable hop limit in the shared packet helper; its previous zero value killed
every packet at the first decrement.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): require stable per-flow translations
8c6ec8d 
A round trip did not prove that FlowLookup reused an installed translation;
a deterministic allocator could return the same tuple by coincidence. Send
repeated packets and distinct flows, requiring one flow to keep its tuple
without sharing it. Mutation checks distinguish table reuse from allocator
selection.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): verify destination selects the correct VPC
d4748a9 
With one peering, any forwarded packet appears to choose the right VPC. Add
three peers with address-encoded prefixes so wrong-table, swapped, and
off-by-one selections are observable. Derive the expected VPC from the
destination chosen by the test instead of consulting the production peering
table.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): compare burst and singleton behavior
7aa41e1 
Every property sent one packet at a time, while the driver supplies bursts
and FlowFilter classifies a whole burst before NAT sees any member. That
ordering can change results. Generate bursts of distinct flows and require
each packet to receive the same treatment it gets when sent alone.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): catch duplicate allocation within a burst
1aa57f4 
Every packet in a burst carries flow state captured before the first packet
reaches NAT. Without a table recheck, one UDP flow consumed a public port per
packet and a TCP SYN followed by data lost the data segment. Drive repeated
packets of one flow together and require the burst to allocate exactly once.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): verify inbound port forwarding end to end
7d940ee 
Routed pipeline properties covered outbound masquerade only, leaving the
externally initiated port-forwarding path untested. Generate requests to a
published port and require delivery to the configured internal host. Compute
the expected tuple from the test's prefix and port offsets rather than the
table used by production.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): assert contracts at stage boundaries
bcf119e 
End-of-pipeline assertions cannot identify the stage that first violated a
packet invariant and can miss errors later stages compensate for. Insert
read-only checkpoints into every routed property and verify decapsulation,
placement, translation readiness, and final state where each contract is
owed. Coverage guards require every checkpoint to run.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(pipeline): reduce stage-count stack pressure
2b9e71d 
The 999-stage test left only a few bytes of stack headroom per packet. Adding
an unrelated field to PacketMeta could therefore abort the test process.
Reduce the synthetic stage count while preserving the chaining behavior the
test exercises.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): enforce one allocation per flow everywhere
6941b50 
A dedicated burst case checked duplicate allocation, but future properties
could generate the same traffic without inheriting that assertion. Add a
stage-boundary invariant that correlates packets before and after masquerade
and requires one allocation per original flow. TestMeta supplies identity
because source translation changes the normal flow key.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): constrain rewritten sources to configured pools
1b20899 
Translation could choose an arbitrary source tuple and still reverse cleanly,
so round-trip properties alone accepted values the configuration never
offered. Whenever masquerade changes a source, require the result to belong
to a declared pool. Leave return-path sources alone because they belong to
the remote endpoint rather than a local pool.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): require deterministic pipeline replay
65854e1 
Bolero shrinking assumes the same input reproduces the same failure. If the
pipeline answers differently on replay, minimization can report a case that
no longer fails and saved cases become unreliable. Run the same fixed
scenario twice and require identical packet results.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): model traffic as stateful conversations
90e3764 
Inline packet generation and judgment worked for one conversation but could
not be paused and interleaved with another. Introduce a Load state machine
that owns packet production, observations, completion, and its independent
oracle. Migrate the strongest round-trip property so failures retain the
conversation history that produced them.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): verify interleaved conversations
1e64ecb 
Conversations that pass in isolation can interfere when one reply shares a
burst with another request. Generate schedules across several stateful loads
and require each conversation to finish satisfied. Count genuinely mixed
polls so a schedule that never interleaves cannot pass while exercising only
the existing sequential case.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): add nonblocking burst traffic
df094b8 
A request-response load exposes only one packet at a time, producing
singleton bursts regardless of the scheduler. Add a blast load that emits
several packets without waiting and can share a burst with other traffic.
Measure the packets each burst actually carries so interleaving guards can
fail when the intended shape is absent.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): derive traffic from validated configuration
b1ee7bb 
Hand-aimed addresses stopped exercising the pipeline once generated overlays
offered different prefixes and ports. Build loads for every expose and
direction declared by the validated configuration while keeping each load's
oracle independent of dataplane tables. Count each expose flavour separately
so omitting a whole kind cannot pass unnoticed.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
feat(config): generate overlays from operation sequences
770237c 
Static builders cannot explore how adds, removals, and peering changes
interact, leaving generated pipeline tests with fixed topology. Implement an
operation algebra for VPCs, peerings, exposes, and flavours, including undo
and read/write footprints. Verify validity, reversibility, noninterference,
commutativity, and reference cleanup, with counters that prevent operation
kinds from going dead.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): exercise generated overlays end to end
c85333a 
The configuration algebra proved its outputs valid, while the pipeline
harness still relied on one written fixture and its hard-coded assumptions.
Draw the overlay, the traffic it implies, and the schedule as one case, then
run every derived load through the production pipeline and its independent
oracle.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland
style(dataplane): remove unused harness imports
4c132c6 
The new pipeline-harness modules left three imports unused, causing the lint
build to fail. Remove them and keep the imports scoped to their callers.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
@daniel-noland
test(dataplane): fail when declared traffic is abandoned
d1a1ec1 
An unchecked load can be valid in general, but loads_for returns only traffic
the configuration declares it can carry. A global checked-count let every
load for one VPC, expose kind, or direction give up as long as another load
completed. Treat any derived load that abandons its conversation as a
property failure.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
@daniel-noland
test(dataplane): make the VLAN control deterministic
71e5dcd 
The VLAN property's positive control depended on both a random configuration
and a random frame being compatible. Instrumented runs drew too few cases
and sometimes delivered nothing despite correct code. Use a known-good inner
packet against the fixed fixture, making delivery structural and keeping the
tagged-frame rejection meaningful.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
@daniel-noland
fix(dataplane,config): use model-aware synchronization
4d8e482 
Direct synchronization imports bypassed the concurrency facade, failing the
repository lint and hiding atomics or LazyLock scheduling points from Loom
and Shuttle. Route these primitives through concurrency::sync so the model
checkers can substitute their implementations and explore the harness state.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
@daniel-noland @claude
fix(config): move algebra counters to process globals
fc0d647 
A static facade AtomicUsize cannot compile with Loom because Loom's
constructor is not const. These counters measure which algebra operations a
Bolero process reached and are not part of any concurrency model. Allocate
them through process_global so Loom builds without treating them as modeled
state.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(dataplane): take the pipeline harness off the model-checking facade
44bf431 
The 37 coverage statics were `LazyLock<AtomicU64>` built through
`concurrency::sync`. Under `--features shuttle` those are shuttle primitives
constructed outside any shuttle execution, so the first `fetch_add` aborts with
"`ExecutionState` is not set" and takes every property in the file with it.

Nothing in this file models concurrency: there is no `#[concurrency::model_test]`,
no `concurrency::stress` and no spawned thread -- it drives the pipeline from
plain `#[tokio::test]` properties and reads the counters once at the end. So the
counters want `concurrency::process_global`, which is what `algebra.rs` and the
mgmt harness already use for exactly this.

The `Arc<Mutex<Translations>>` is deliberately left on the facade: it is a
runtime value inside the harness rather than a `static`, and the two have
different failure modes.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
test(pipeline): record the stack budget the stage-count canary asserts
fe4159e 
`long_dyn_pipeline` is a stack-headroom canary -- `DynPipeline::process`
recurses once per stage -- and it is what caught `PacketMeta` growth eating that
headroom. Lowering it from 999 to 500 answered the canary without writing down
what the new number guards, so the next person to hit the limit has no way to
tell a real regression from another trim.

Say that 500 frames is the budget, and that a stage frame past roughly 4 KiB
overflows it again, so the next reduction buys less than this one did.

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@daniel-noland @claude
docs(fuzz): point the corpus documentation at the directory actually …
7bbf64b 
…used

`just fuzz` still told readers that findings land in a `__fuzz__` directory
beside the test. They land under `fuzz_corpus_root` -- `.fuzz-corpus` unless
`FUZZ_CORPUS_ROOT` overrides it -- since the coverage corpora were isolated from
the unit tests earlier in this same chapter.

The `.gitignore` entry for libfuzzer's per-worker logs also lost the second line
of its two-line comment, leaving it ending on "when".

Signed-off-by: Daniel Noland <daniel@githedgehog.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@daniel-noland daniel-noland

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@daniel-noland