fix(fspy): replace quiescence locking with crash-tolerant shared-memory publication - #675
Open
wan9chi wants to merge 51 commits into
Open
fix(fspy): replace quiescence locking with crash-tolerant shared-memory publication#675wan9chi wants to merge 51 commits into
wan9chi wants to merge 51 commits into
Conversation
…cation The IPC channel previously required writer quiescence before reading: a file lock (or #577's active-writer gate) had to drain before the receiver could parse the inline frame stream. A traced process that closed the lock descriptor while keeping the mapping writable corrupted parsing (#544), and one that never exited (a daemon) or died mid-record could block collection or poison the writer count forever. The shared memory now uses a two-ended layout: an allocator word admits claims and closes the channel, a descriptor table grows from the front, and payloads grow from the back. Each frame commits by publishing its descriptor with a release CAS; closing atomically aborts every unfinished slot and copies committed payloads out with relaxed atomic loads, so the receiver never waits for a writer, never trusts payload bytes for traversal, and never holds a reference into memory another process may mutate. Loss of a record by a live writer (capacity, abandonment) flags the trace incomplete so the run is not cached from an under-reporting trace; process death needs no cleanup because records are published before the recorded operation is performed. Closes #544. Supersedes #577. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fspy benchmarklinuxmacoswindows |
Temporary stderr phase timings to locate the CI launch regression on the Linux benchmark runner. Will be dropped before merge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This reverts commit d2fa1de.
…path The crash-tolerant close moved two hidden costs into the tracked child's launch window on journalling filesystems: the first write to the sparse 4 GiB backing file (a millisecond-scale block allocation, previously paid lazily or never) and unmapping the receiver's view (previously after access collection). The Linux benchmark runner priced them at ~2.2 ms and ~0.6 ms per launch. Pre-fault the header page on a background thread at channel creation — a protocol-neutral compare-exchange of zero with zero, run concurrently with process startup — and release the receiver's mapping on a detached thread after the frames are copied out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the packed allocator word and its compare-and-swap loop with two monotonic counters over a fixed table/payload partition. A claim is two wait-free fetch_adds validated against the fixed region bounds; failed claims overshoot the counters harmlessly because committed descriptors are self-describing and readers clamp to the region capacities. The close boundary becomes a snapshot load: claims that arrive later land in slots the receiver never visits and are dropped under the same publish-before-perform argument as freeze-race losses. The CLOSED gate — whose write materializes the counter page, a millisecond-scale first-block allocation on journalling filesystems when the trace is empty — moves onto the deferred teardown thread, which lets the pre-fault machinery from the previous commit be deleted outright. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The wait-free rework deleted the pre-fault thread on the theory that a write-free close no longer needs the page. The benchmark disagreed: on the Linux runner the first touch of the sparse backing file costs milliseconds whether it is a write (a sender's first claim) or a read (close's snapshot), so Linux launches regressed right back. Windows meanwhile improved once the thread was gone — its first touch is cheap and the spawn was the cost. Restore the concurrent header-page warm-up, gated to Linux. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… a thread Reserving one block at the header and one at the payload-region start with fallocate(KEEP_SIZE) is a cheap metadata-only operation at channel creation, so the milliseconds of journalled block allocation that some filesystems charge for the first touch of each area no longer need a background thread to hide them — and the payload area, which the thread could not safely touch, is now covered too. KEEP_SIZE because growing the file would desynchronize mapping sizes across processes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lting on a thread" This reverts commit 528d1d1.
The fallocate experiment showed the first-touch cost is in the fault path, not block allocation, so only a real touch helps. Give the pre-fault thread a second target: a protocol-owned warm word between the table and the payload data, so the page where the first payloads land is materialized without racing any writer's payload bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This reverts commit c4ba8dd.
…opying Frames now owns the mapping and lazily hands out per-span borrows of the validated committed payloads. A committed span is immutable under the protocol and disjoint from everything a live writer may still touch, so the borrows are sound without a copy; the trust argument lives in the reader module docs. This also collapses the close-time machinery: the CLOSED gate returns inline into close (its page is pre-warmed on Linux where first touches are expensive), the deferred-teardown thread is gone, and the mapping is released when Frames drops — naturally off the collection path. The receiver-side frame validation pass in the supervisor is dropped with it; committed frames are complete by protocol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nothing is collected anymore — frames are borrowed in place — and the async wrapper descended from the file-lock era, when acquiring the trace could block until every sender exited. Closing is now bounded by the number of reported records and runs inline, so the type becomes ChannelAccesses with a TryFrom<Receiver> conversion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
wan9chi
force-pushed
the
claude/fspy-shm-publication-design-76917a
branch
from
August 15, 2026 01:05
e560a84 to
0a80d2b
Compare
The protocol layer should not know its consumer: describe the publish- before-perform rule as the intended usage contract and the incomplete flag as a property of the channel, with no mention of what sits on top. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The descriptor table's length becomes the protocol's const-generic parameter, so the header and table are one repr(C) struct — offsets become field accesses, the table a real array, and the per-accessor unsafe pointer derivations collapse into one borrow. The payload area stays outside the struct deliberately: writers hold exclusive borrows into it that must not alias the shared region borrow. The channel names its layout the same way: channel::<SLOTS>() sizes the backing file to capacity_for_slots(SLOTS) (the exact inverse of the slots_for_capacity sizing rule), every process names one shared SHM_SLOTS constant, and a sender now rejects a region whose size disagrees with the layout instead of panicking inside geometry assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The const-generic table size taxed every signature and call site, and it bought a property the mapping already provides: with the layout derived from the mapping length alone, the region is self-describing — writers and the receiver compute identical bounds from the size of the file they mapped, with no shared constant to agree on and no size handshake to get wrong. What the struct experiment taught survives: one unsafe borrow now builds three typed views — the repr(C) header, the descriptor table as a slice of atomics, and the raw payload area — so counters are named fields, slots are bounds-checked indexes, and only payload spans remain pointer arithmetic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The incomplete flag had one real writer — capacity exhaustion — and the counters already record that: a failed claim's bumps push a counter past its limit, counters never move backwards, and the bump precedes the operation whose record was lost, so the close snapshot either sees the overshoot or the loss belongs past the boundary. The flag's other writers were bug-only paths. So the flag word, FrameMut's Drop, and the write_encoded flagging wrapper are gone; abandoning a frame now leaves exactly what dying does — an unfinished slot the receiver ignores — and acting on an abandoned record is outside the usage contract. Oversized frames become an asserted precondition: a caller error, and the one loss counters could not record. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The protocol had two ways to write a record — write_encoded, with an error enum only one caller half-used, and the hand-rolled claim/serialize/finish sequence in the Unix client. Both callers want the same thing: serialize the record into a frame and skip it on any failure, because an intercepted call must proceed no matter what. That helper now lives once, on the channel's Sender; shm_io keeps only claim, fill, and finish. Also: a plain-words README section on how a full region is handled, and a mermaid dependency graph of the module files as a reading order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The six-way split was designed around machinery the simplifications have since deleted, and its narrative had drifted: writer and reader derive their own payload references, so state was never the only file touching shared memory. Merge to the boundary that still earns its keep — pure integer math versus code that touches the mapping: - layout.rs absorbs the descriptor codec (both plain arithmetic) - shared.rs is state + writer + reader in reading order, with the reservation types and SharedState going private to it - mod.rs keeps the surface, overview docs, and integration tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completeness by counter overshoot had two holes. A frame larger than a descriptor can describe (> i32::MAX bytes) could not be reported at all, so claiming one panicked — reachable in the preloads, whose record lengths come from path strings the traced program controls, breaking the promise that a preload never panics its host. And a writer killed inside a failed claim left an overshot counter behind, marking a channel incomplete over a record whose operation never ran. Replace the overshoot rule with a loss flag in the header's reserved space: every failed claim stores it before the writer moves on, and the receiver reads it once at close. The non-overflow path is untouched — the flag's cache line is only written by a claim that is already failing. The report-before-perform order gives the same rule-1 guarantee as commit-before-perform: a report the receiver misses belongs to an operation performed after close, and a writer that dies before reporting never performed the operation at all. Oversized frames now fail like any other refused claim, without poisoning the counters, so the channel stays usable for the records after them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HEADER_LEN predates the typed header: when layout.rs was offset math only, there was no struct to measure, so the size was a literal and a const assert tied the struct to it after the fact. Move the Header struct into layout.rs — it describes the region's shape, which is that file's job — and derive HEADER_LEN from size_of. The sizing math now follows the struct automatically; the one remaining literal is an assert pinning the header to a single cache line, which is a design intent no struct can express. CLOSED moves along with it, keeping all the bit meanings next to the slot codec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every one of these is a leftover from a deleted design, kept alive only by habit or by tests: - ReserveError duplicated ClaimError variant for variant; try_claim now returns ClaimError directly and the mapping in claim_frame goes away. - Reservation was a named pair passed once between two functions in the same file; a destructured tuple says the same thing. - SharedState::mapping_len wrapped a field its one caller can read. - The close/pre_fault wrappers in mod.rs re-stated shared's docs to delegate one call; the functions are now re-exported like the rest of the surface, with the wrapper's doc text folded into the real ones. - Sender's Deref to ShmWriter served only tests, which now reach the writer field directly; FrameMut, ClaimError, and ProtocolError are no longer nameable outside the channel (production never names them), the error types staying test-visible for assertions. - into_memory is gated to the non-miri test that is its only caller. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SlotState classified every slot value the receiver could read, but its only consumer treats all invalid classes identically, and the validation that followed already refuses every pattern the classification singled out: an unfinished zero decodes a zero length, and any value carrying the aborted bit decodes a length beyond the 31-bit limit. Collapse decode and validate into one step returning Option<PayloadSpan>; the freeze loop keeps a slot's span, skips ABORTED, and calls everything else corrupt. The three-state table stays as the comment documenting what slot values mean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The free unsafe close made the module's unsafe boundary uneven: the writer pays its contract once at attach and operates safely, while the receiver re-asserted the same contract at every close call — in the channel, far from the creation-time facts the SAFETY comment cites. ShmReceiver mirrors ShmWriter: one unsafe constructor with the identical contract, eager geometry validation, and a safe consuming close. The channel constructs it where the region is created, so Receiver::close is now safe code. pre_fault stays a free function: it is a creator-side warm-up on a throwaway view, belonging to neither endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mod.rs stopped earning its keep as a separate facade: the wrapper functions are gone, the curated surface decayed into verbatim re-exports, and the boundary itself manufactured the cfg(test) re-export of the error types, needed only so tests one file away could name them. Merge the mechanism into the module root — overview docs, region views and ordering contract, writer side, receiver side, then the tests — and keep the boundary that still means something: layout.rs describes the region, mod.rs operates on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SharedState accumulated every protocol operation while the writer and receiver types just forwarded to it — but no operation was actually shared between the sides. Each now lives with its only caller: claim_frame absorbs the claim sequence, the loss report, and the payload pointer math (whose base-offset round-trip cancels out once merged); finish holds the commit CAS; close holds the snapshot, the gate, and the freeze pass; pre_fault holds its warming CAS. SharedState is reduced to what the sides genuinely share: the borrowed views and their one unsafe constructor. The inlining surfaced one behavior the old freeze helper hid: a second close over the same region finds slots already aborted in the CAS failure path, so the ABORTED skip stays (commit_after_abort test). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The receiver type existed only to be consumed by its one method, and Frames was that method's result — two names for the halves of a single transition. ShmReader is the whole thing: the unsafe close constructor attaches to the region (the writer contract), snapshots, gates, and freezes, and the value you get back is the reader you iterate. The name pairs with ShmWriter the way the original protocol's reader did. Iteration becomes a real iterator type: Iter walks the validated spans, and &ShmReader implements IntoIterator, so for-loops work directly — clippy immediately insisted the tests use them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The loss flag and the CLOSED gate answered the same question from two sides — is this channel still worth writing to? — so a failed claim now sets the gate itself. One bit is both the loss report completeness derives from and the valve that stops writers: the boundary, the gate, and every loss report sequence in a single word's modification order, which tightens rule 1 from a two-location argument to one. Consequences, all deliberate: - The first lost record condemns the channel. Later records would ride a result the receiver must already reject, so refusing them is the same economy the gate always bought. - This also closes the payload-counter wrap hole: wrapping requires billions of failed claims, any failure sets the gate, and the gate refuses every claim before a span is built — no rollback needed. - A second close over the same region now reports incomplete, which is the truthful verdict: a re-close cannot vouch for records refused since the first gate. - The header returns to two counters, and the receiver's completeness verdict rides the snapshot load it already performs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…spans Closing allocated a Vec of every committed span — the one allocation in the module, and O(frames) memory that a multi-million-record trace turns into real heap. It was never necessary: the freeze pass already makes the snapshot's slice of the descriptor table immutable, so the reader only needs the mapping, the frozen prefix length, and the frame count. Iteration re-reads the terminal slots and re-decodes each committed descriptor — pure math on bits close already validated, so the iterator stays infallible. Payload visibility travels with the reader: whatever transfer carries it across threads carries the freeze pass's Acquire along (rule 3). shm_io is now allocation-free end to end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SharedState was re-derived from the mapping on every operation because a struct cannot store references borrowed from a field it owns. But this was never true self-reference: the views point into the mapping's target, which the attach contract already requires to be address-stable, not into the endpoint value that moves. NonNull fields express exactly that, so SharedState loses its lifetime parameter and both endpoints construct it once and store it — the geometry asserts run once per attach, and state()/re-borrows disappear. Unsafe relocates rather than grows: three one-line accessors carry the pointer-to-reference step, Iter construction loses its unsafe block, and close loses its borrow-scoping braces. Raw fields cost the endpoints their auto Send/Sync, now stated manually with the justification that was implicit all along: the protocol synchronizes every access, and the views point at stable, independently owned memory. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single protocol module mixed three audiences: writer code, reader code, and the definitions both must agree on. Split along that line: - writer.rs — claim, fill, finish - reader.rs — close, completeness, iteration, and the borrow argument - layout.rs — only what both sides share: the shape math, the header, the slot codec, the ordering contract, and the views struct, renamed SharedState -> MappedLayout: it holds no state — it is the layout bound to one concrete mapping. Endpoints store it as 'mapped'. - mod.rs — the surface, overview docs, and integration tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MappedLayout kept the mapping length so decode and the writer's offset math could re-derive the payload bounds — a leftover from when the views were rebuilt from the mapping on every call and the length was the seed. The stored views already carry both bounds verbatim: the table's length is max_slots, so payload_base is one multiplication away, and the payload slice's length is payload_region_len itself. Re-sign validate and decode to take those bounds, let MappedLayout hand them over (mapped.decode(bits), mapped.payload_base()), and the third copy of the information disappears — along with Iter's mapping_len field and the 'not derivable from the parts above' caveat, which guarded a re-derivation nothing performs anymore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every branch in the sizing math served lengths nobody wants: the eight-slot floor and its cap exist for regions under ~576 bytes, the zero-payload case for under ~100, and the payload round-down for lengths that are not a multiple of 8 — degenerate shapes exercised only by the tests that verify the branches handling them. The protocol gets to choose its supported lengths, so choose regular ones: multiples of 8 between 1 KiB and 4 GiB, checked in one place (is_supported, asserted at attach, guarded by senders) and produced in one place (the channel rounds its requested capacity up via round_up_region_len). The sizing rules collapse to branchless arithmetic — the table is an eighth in one division, the payload region is one subtraction, and alignment holds by construction instead of by correction. Every length in real use produces a bit-identical layout; only sub-KiB and unaligned regions go from degraded to refused. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…alves Descriptors stored offsets measured from the start of the mapping while everything else already worked in payload coordinates: the counter hands out payload-relative offsets, the writer added payload_base only to encode, and the reader checked the base bound only to subtract it again. Encode what the counter returned: the conversions disappear, validate loses a parameter and its lower-bound check — in payload coordinates no bit pattern can name the header or the table, so the invalid states are unrepresentable rather than rejected — and MappedLayout's payload_base and base accessors go with them. The payloads view now carries both the location and the bound of the only area descriptors can point into. Also unshare layout.rs down to what both sides genuinely use: committed moves to the writer, decode/validate/PayloadSpan/ABORTED to the reader (with their tests), and layout keeps the format both sides agree on — the field layout, UNFINISHED, and the bit-field constants. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'close' pulled double duty: the receiver's one-shot operation and the gate state any failed claim can set. Since the gate gained its second trigger the words have named different things, so split the vocabulary: 'seal' is the act — ShmReader::seal snapshots, gates, freezes, and returns the reader — while 'closed' and the CLOSED bit stay the state, reachable by seal or by an overflowing claim. It also sheds close's end-of-lifetime connotation on what is, for the reader, a constructor. The channel layer's Receiver::close keeps its name: at that API the receiver closing the channel is exactly what happens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The table length becomes a const parameter and the header and table become one repr(C) struct — Meta<SLOTS> — that the channel instantiates (SLOTS = 1 << 26, matching what the eighth rule gave the 4 GiB production region). The payload area is simply the rest of the mapping, so the runtime geometry collapses to one number, size_of::<Meta>(), and attaching checks a single bound: the struct must fit the mapping, leaving a payload area the descriptors' 32-bit offsets can address. The sizing rules, the supported-length predicate, and the eighth-rule arithmetic are all gone; channel capacity now means payload bytes, with the fixed struct added on top. Unlike the earlier const-generic round, nothing needs to derive one const from another and no exact size handshake exists — senders accept any mapping the struct fits into — so the parameter stops infecting call sites beyond the channel's own aliases; fspy is untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload offsets were u64-aligned because the original protocol stored an AtomicI32 size header inline at the start of every frame — atomics must be aligned, so frame boundaries had to be. The descriptor redesign moved every typed thing out of the payload area, leaving untyped bytes nothing dereferences, but the rounding survived on a circular justification: reservations were rounded so offsets stayed aligned, and validation checked alignment because rounding guaranteed it. Claims now reserve exactly the frame's length: reserved_payload_len and SLOT_LEN are gone, validation is a length range and one bound, and up to seven padding bytes per record return to the payload budget. Spans may be byte-adjacent — disjoint &mut [u8] ranges are race-free at byte granularity, which miri now exercises via the odd-length concurrent tests. Two more fossils from the same sweep: ProtocolError kept its enum shape from a deleted second variant — now a struct — and fspy's SHM_CAPACITY comment predated capacity meaning payload budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Payload-budget capacity existed to keep small test capacities working under the huge compile-time table, but region-length capacity is less code — the channel fail-fasts through the existing supported-length check instead of computing a file size — and it puts the production region back at exactly 4 GiB with the same ~3.5 GiB payload area the eighth rule produced, so CI and the benchmarks compare like for like. Channel tests pass a sparse GiB instead of a few KiB. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Header existed to be reached through: every consumer immediately took .claims or .payload_reserved out of it. Meta is now literally the protocol's fixed part — two counters, then the slots — and MappedLayout exposes claims()/payload_reserved() directly. The _reserved padding goes with it. Its compatibility half was a fossil of file-format thinking: both endpoints compile from one crate and a region never outlives its channel, so there is no version boundary to reserve space across. Its cache-line half shielded only the first six slots' commits from the counters' line, for the first microseconds of a channel that then runs for a whole build — speculative padding of the kind this series has been deleting. If a benchmark ever shows the false sharing, one padding field brings it back. Checked and kept in the same sweep: the channel-level Send/Sync impls (Windows Mapping has no auto impls, so they are the cross-platform seam), and the infallible-on-64-bit try_from guards (the 64-bit-target boundary, not dead defensiveness). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The frame limit was a named constant compared against on both sides, then converted anyway — validate-then-convert where the conversion alone answers the question. The 31-bit length field's honest spelling is i32::try_from: the writer's oversize check becomes that conversion (MAX_PAYLOAD_LEN deleted), the offset conversion narrows to u32 — the actual field width — and committed's arguments become the field types, replacing its debug_asserts. On the reader, decode reverses the same conversions: one i32::try_from refuses the aborted bit, an unfinished zero, and oversize alike, and PayloadSpan::validate dissolves into decode, its tests re-expressed as wire values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The i32 detour existed only because ABORTED sat on bit 63, stealing one bit from the length field above it. But the aborted value is the protocol's own choice, and any value with a zero length field is unmistakable — committed lengths are never zero — so ABORTED is now 1: a zero length with offset one, which no writer commits. With bit 63 freed, the descriptor is simply two u32 halves. The writer's oversize check is u32::try_from, decode's extraction is two truncations and a zero test, PayloadSpan stores the field types themselves (usize appears only at the pointer boundary), and the frame limit rises from 2 GiB to u32::MAX bytes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fields drop in declaration order, and what borrows must die before what is borrowed. MappedLayout has no drop glue today, so the old order was merely fragile rather than wrong — this makes the endpoints correct by construction instead of by that accident. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
is_supported_region_len was a precondition API: call it first or the constructor panics. The constructors now answer for themselves — MappedLayout::new and ShmWriter::new return None for a region that cannot host the protocol, seal reports ProtocolError::UnsupportedRegion (the enum's second variant is real again), and pre_fault quietly does nothing, since a region nobody can attach to needs no warm-up. The alignment check is the pointer conversion itself: a stable stand-in for the still-unstable <*mut T>::try_cast_aligned, alongside NonNull::new. The predicate is deleted. Senders map None to an error instead of guarding; the channel fail-fasts at creation by performing the same fallible attach a sender would (over &Mapping — AsRawSlice now has a reference impl), so a bad capacity fails the task before any child spawns. The misaligned-region test asserts a None instead of catching a panic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The expect(dead_code) machinery — including the cfg_attr keyed on both test and miri — existed only because into_memory read the writer's owner field in some build configurations. into_memory itself was a workaround for ownership that the AsRawSlice reference impl now solves: its one caller, the killed-writer test, borrows the mapping for its surviving writer and hands the mapping itself to the seal. With the last read gone, both owner fields are plain _mem, and the underscore says everything the lint attributes said. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A slot value is now parsed once into what it means — SlotState, either Unfinished (nothing published: never finished, died, abandoned, or frozen by the seal) or Committed with a NonZeroU32 length — instead of shift-and-mask helpers on both sides. The nonzero length is the wire invariant as a type: it cannot collide with UNFINISHED or FROZEN, and NonZeroU32::try_from(frame_size) makes the writer's oversize check, zero-exclusion, and field-width conversion one step. bytemuck casts the u64 to its two u32 halves (same machine, native byte order), and the bounds check lives with the two readers of a span. Reviewed while finishing: CLOSED stays bit 63 rather than u64::MAX — stragglers keep fetch_adding after the seal, and an OR-ed bit survives 2^63 increments while an exact sentinel is destroyed by the first, silently reopening the gate; and the payload area keeps its u32 bound (now a conversion), without which the writer's offset conversion could panic on >4 GiB regions. ABORTED is renamed FROZEN to match the seal vocabulary. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gate's correctness rested on a patience argument: nobody performs 2^63 claims over a channel's lifetime, so increments never carry into bit 63. Replace it with a state invariant: every refused claim subtracts its counter increments back, always after the gate is set — observed in the add's return, or installed by the refusal itself. Subs pair with the same claim's own adds, so the counter never drops below the successful count or any seal snapshot, and never rises past the table length plus one per claim in flight. Reaching the bit by counting would now take 2^63 simultaneous claims, which a 64-bit address space cannot host — correctness follows from the machine model the module already asserts, not from how long overflow takes. The undo needs no loop because it is contention-free by construction: a claim owns its own +1, and fetch_sub never fails. Gate-before-sub is what keeps the undone values harmless — the gate lives in the same word, so every read of a subtracted count carries the bit with it. The payload counter takes back only out-of-bounds reservations (no live span sits above those); in-bounds reservations of refused claims stay counted, never materialized and capped by the region. The one residue wait- freedom cannot erase is a writer dying between its add and its undo: one count per death, 2^63 deaths to so much as condemn the channel spuriously. Hot path unchanged: a successful claim is still two unconditional fetch_adds. Tests peek the raw counters to pin the invariant down. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shorter sentences, no arguments against designs the code no longer contains, and a few fixes the pass surfaced: the oversize comment still promised the channel keeps working after a refusal (the gate condemns it), the codec comment said header where the region has counters, the writer's struct doc pointed at an ordering contract 'above' that lives in layout, and the reader's module doc opened with a stray blank line and 'closing' where the operation is the seal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Collecting file-access traces required writer quiescence: the receiver could only parse the inline frame stream after every sender released a file lock. That coupling broke in two real ways:
python-daemon) that closes inherited file descriptors releases the lock while keeping the shared-memory mapping writable. The reader then races live writers and interprets path bytes as frame headers, panicking the runner.SIGKILL/crash.Correctness must not depend on file-descriptor lifetime or on writers running cleanup during teardown.
Design
The channel (
fspy_shared::ipc::channel::shm_io) now publishes frames through a descriptor table instead of inline headers:Consequences elsewhere: the lock file is gone (
ChannelConfnow carries only the shm id),Receiver::lockbecame consuming nonblockingReceiver::close, the ouroboros self-referencing guard infspy::ipcis replaced by owned copied-out frames, and the preload clients treat claim failure as skip-and-forward — a preload library never panics its host process.The module structure keeps each safety argument local:
layout/alloc_word/slotare pure functions,stateis the only module touching shared memory (and documents the three-rule ordering contract),writerandreadereach carry one aliasing justification.Verified with miri across the protocol tests (state machine, close races, concurrent writers), plus cross-process tests including a hard-kill writer (
SIGKILL/TerminateProcessviaChild::kill).Closes #544. Supersedes #577.
🤖 Generated with Claude Code