Libmdbx spike - #105
Open
mo4islona wants to merge 14 commits into
Open
Conversation
The LSM disk bill is the driver: ~640 MB/s device writes against a 62 GB rolling window on a production replica, with the measured 4.03x LZ4 ratio on CF_TABLES making app-level page compression a precondition for any win. Decision is spike-gated: port only if device writes drop >=3x without head-freshness regressions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drives the hotblocks write shape (chunk commits, chunk merges, retention trims) against libmdbx and a production-tuned RocksDB reference. The sqd_storage::kv traits are implemented over libmdbx and exercised for real (writes go through KvWrite, a post-run scan decompresses every live page via KvReadCursor) — the seam the ADR claims ports verbatim. Write amplification needs /proc/self/io, so decision numbers come from a Linux run; macOS gives latency and footprint only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Free-running without a durable point parked every freed page: the file reached 110x the live data. A 1s mdbx_env_sync cadence holds it at one growth step, at the cost of fsync landing in commit tails - the trade to quantify on Linux, where fdatasync is ~1ms against F_FULLFSYNC here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unblocks signet-libmdbx (the published reth-lineage mdbx binding; its read-tx-timeouts feature keeps the GAP-29 slow-reader abort) for the ADR 0002 port. Not 1.92/1.93: both fail release codegen of sqd-array with E0275 (overflow evaluating `impl ArrayWriter: ArrayWriter` — the recursive shift()/ArrayWriterView opaque chain; dev and check pass, so CI would not have caught it). Fixed upstream by 1.94. Verified on 1.94: release workspace build, 318 tests, clippy correctness gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follows the flush-bench pattern: an image stage per docker.yaml target, run as a Job pinned to an idle prod-class NVMe node. Adds cgroup io.stat sampling — /proc/self/io write_bytes counts page dirtying, not proven device traffic (see the 2026-07-16 flush-bench measurement), and the write-amplification verdict must come from the authoritative counter. The Dockerfile base moves in lockstep with rust-toolchain.toml because the toolchain file is not part of the build context. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gate came down to page size: 4 KiB writes 3.7x less than the rocks reference and passes, 64 KiB writes 1.4x more and fails. The wave runs close the reclaim question: the mdbx file is a permanent high-water mark (~1.35x peak live; tail truncation never fires even with shrink_threshold armed) but reuse holds — zero post-wave growth at paced rate — while rocks buys its reclaim back for 3.7x the device writes. Two constraints worth the price of the runs: mdbx_env_sync racing SafeNoSync commits aborts the process, so durable sync is serialized with writes; cgroup io.stat double-counts stacked md/dm devices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The wave runs close the reclaim question the spike left open, and the answer is architectural: any engine that returns space online pays the compaction multiplier this ADR removes, so reclaim moves to the env boundary. The policy generalizes the shipped NET-896 position cap — ring past the floor, coverage gap instead of unbounded growth — to an engine-enforced byte quota, with env-granular reset as the defrag mechanism. The spec letter conflicts in seven places; amending them is Stage 0 of the port plan and the gate for flipping this ADR to accepted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…R-6) Stage 0 of ADR 0002, and a reconciliation: PR #77 shipped the Api.max_blocks position cap five days after the last audit, so the External guarantee row was already stale and its downward-instruction clamp contradicted the N3 resolution. RS-13 makes the space bound part of policy semantics — effective trim bound for self-healing policies, write-pause at quota for promise policies, P-REORG-KEEP ring floor for both; RS-14 names the online dataset storage reset that returns high-water slack; RS-6 splits into a hard quota and an amplification ratchet against peak live. GAP-42/43 record what the implementation still lacks: gap-mode observability and any byte bound at all. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… reality The downward-instruction clamp now matches production: clamp_floor keys on the position cap's presence, not its governance, while the byte bound alone conditions on governance so the WP §2.5 RESET stays reachable once every dataset carries a quota. The watermark is defined against occupied pages because the file is a permanent high-water and would pin gap-mode after one peak; an emergency trim that cannot commit (deletes need COW pages at a full map) degrades to the FM-STOR-6 pause instead of a crash loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The plan leaned on things that turned out not to exist or not to hold: the published signet binding ships no read-timeout, mdbx deletes can themselves return MAP_FULL, data pages carry no checksums (FM-STOR-4 would regress), multi-commit table builds keep needing the orphan journal on any engine, and /ready has no per-dataset readiness for RS-14 to stand on. Reads get their own cold-cache gate, and the accepted flip now waits on a Stage 1 vertical slice instead of Stage 0 alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A swallowed periodic-sync error would silently unbound file growth and falsify a run, so the syncer panics now. The 1.35x sizing rule holds only at paced rate (free-run bursts park a sync-cadence window of churn on top) and the stale-reader projection had quoted the 64K single-env device rate. Decoder shoot-out on the decision Xeon settles the Stage 1 choice: C lz4 into a reused buffer is 5.8x over the lz4_flex safe-decode the spike used; disabling safe-decode buys only 2x. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The rocks reference ran an uncompressed WAL and default background jobs; at production pace that overstated its device writes by ~40% -- the honest paced advantage is 3.3-3.8x, not 5.5-6.3x (free-run barely moved: 3.54x, gate holds). Inline merges never contended with commits for the writer lock; --concurrent-merges restores the compaction_loop shape and prices the head stall at one merge-txn hold (p99 323us, max 14ms vs rocks 60ms). ADR follow-ups from the same review round: production runs buffered IO (the direct-io claim was false), so porting note 2 becomes the memory-regime story; DP-2 gets the two-signal watermark; the binding is decided (vorot93 0.6.x + FFI shims); hash indexes on internal are an unpriced random-insert stream (note 4); the measured prod scan bracket bounds decode at ~0.15 core/pod and moves the cold-cache gate to rollout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Global hash-keyed tables are a no-go under mdbx: at internal's shape (300 entries/commit against a preseeded 1M-entry tree) the index stream costs 27x the data stream (+31.2 GB vs a 1.17 GB baseline; rocks pays +4.0 GB for the same) and inverts the engine decision on hash-indexed datasets; write-behind batching at K=16 recovers only 26% and buys 26 ms flush tails. ADR porting note 4 and Stage 2 now mandate per-chunk index tables with fan-out lookups. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…or art Records why the global table fails (random keys x page-unit COW: ~600 leaf rewrites per commit for 26 KB of payload) and the replacement: the index becomes one more immutable table of the chunk — append-shaped writes, deletion and reorg cleanup inherited from the chunk lifecycle, O(chunks) fan-out lookups with a bloom escalation behind a trigger. reth's mitigations (ETL bulk sort, tip-only trickle, no deletes) do not transfer to a rolling window that lives at the tip permanently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.