perf(start-client-core): O(1) buffer drain in client frame decoder#2
Closed
anonrig wants to merge 1 commit into
Closed
perf(start-client-core): O(1) buffer drain in client frame decoder#2anonrig wants to merge 1 commit into
anonrig wants to merge 1 commit into
Conversation
The frame decoder dropped consumed chunks from its buffer with bufferList.shift(), which is O(n). When a single large frame (e.g. a big RawStream payload) is assembled from many small network reads, the extract loop calls shift() once per chunk, making reassembly O(n^2). Track the first un-consumed chunk with a head pointer and advance it in O(1) instead of shifting. Consumed slots are released for GC, and the buffer is compacted when fully drained (O(1) reset) or once the consumed prefix grows past a small threshold (amortized O(1) per chunk). A micro-benchmark draining 1000 small chunks is ~11x faster.
Owner
Author
|
Superseded by upstream PR TanStack#7663 (opened against TanStack/router). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replace the O(n)
bufferList.shift()in the client-side frame decoder (packages/start-client-core/src/client-rpc/frame-decoder.ts) with an O(1) head pointer.extractFlattened()dropped each fully-consumed chunk from the front ofbufferListwithshift(). When a single large frame (e.g. a bigRawStreampayload) is assembled from many small network reads, the extract loop callsshift()once per chunk — and eachshift()re-indexes the whole array, so reassembly degrades to O(n²).This PR tracks the first un-consumed chunk with a
bufferHeadindex and advances it in O(1) instead of shifting.readHeader()reads frombufferHeadas well. Consumed slots are released for GC, and the array is compacted:bufferHead === bufferList.length) → reset in O(1) (the common terminal state), orsplice()it off (amortized O(1) per consumed chunk).This mirrors the existing index-pointer approach already used in
transformStreamWithRouterfor the same reason.Why
Same hot path as the sibling zero-copy PR: decoding streamed server-function responses and
RawStreampayloads. The O(n²) bites specifically when one frame spans many buffered chunks.Standalone micro-benchmark (Node, median of 12 runs), draining N buffered chunks:
shift()Tests
frame-decodersuite passes.CHUNKpayload delivered one byte at a time (forces the header slow path + many whole-chunk consumptions + the fully-drained reset),splice()prefix drop).Notes
extractFlattened, so whichever merges second will need a trivial rebase.import/orderlint errors insrc/client/hydrateStart.ts(virtual-module imports) are unrelated to this change and present onmain.