fix(runtime,desktop): order same-revision catalog reads by a run epoch - #5741
Conversation
|
No description provided. |
hqhq1025
left a comment
There was a problem hiding this comment.
I reviewed the 12-file change that adds a per-session run epoch to runtime catalog projections and uses it to reject older same-revision rows. I found two actionable issues, detailed inline: the new test does not compile (P1), and coordination runs can change their visible running-turn set without changing the epoch (P2).
The current PR is also conflicting with main in packages/runtime-host/src/protocol/index.ts: main already uses compatibility epoch 190 for a different protocol change, so the conflict resolution must preserve both changes and assign a new compatibility epoch. Only lifecycle and label have completed successfully on this head; I could not verify the required test gate. Locally, Node 24 npm run build:test fails at the new test, before focused tests can run. There are no schema or migration changes. This is not ready to merge until the compile failure, run-ordering gap, merge conflict, and required checks are resolved.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
| test('the session run epoch bumps on every turn start and end', async () => { | ||
| const store = memoryStore(); | ||
| const backends = new BackendRegistry(); | ||
| const backend = new BlockingBackend(SESSION_ID); |
There was a problem hiding this comment.
P1: BlockingBackend requires a second options argument (see its constructor at line 554), but this new test passes only SESSION_ID. With Node 24, npm run build:test fails here with TS2554: Expected 2 arguments, but got 1, before any focused runtime test runs. Pass an options object and rerun the build/test gate.
| } | ||
| active.activeRuns.set(run.runId, run); | ||
| active.turnToRunId.set(run.turnId, run.runId); | ||
| this.#bumpSessionRunEpoch(active.sessionId); |
There was a problem hiding this comment.
P2: This epoch only advances when a run enters/leaves active.activeRuns, but runningTurnIds() also includes coordination runs from executionClaims (activeRunsFor, lines 2173-2181). runCoordinationOperation() marks its claim as hostOperation and attaches the run before reservation; after the run unregisters, the claim is released later. Both transitions can change the returned turn IDs without advancing this epoch. A catalog read with the turn still present and another with it absent can therefore have the same revision and epoch; if the older response arrives last, isStaleSummary accepts it and restores the wrong running state. Advance the ordering token when the claim changes the visible set, and test the coordination-run race.
462d38e to
ff7d6a4
Compare
|
Both findings are fixed on the rebased branch (head
On testing the coordination-run race: driving |
c1b5ad3 to
fbe79bb
Compare
hqhq1025
left a comment
There was a problem hiding this comment.
I re-reviewed the 14-file change at the current head. The prior test-construction error is fixed, coordination-claim transitions now advance the epoch, and the protocol compatibility epoch is rebased to 192. The current test check succeeds; after applying the repository's dependency patches locally, npm run build:test and 121 focused tests also pass. The branch merges cleanly with current main.
I found one remaining runtime ordering issue (P2) and one unrelated test-coverage regression introduced during the rebase (P3), detailed inline. The P2 can leave a session row stale across a Host restart, so I would not treat the green checks as a merge decision. I did not run a real Host restart or cross-platform Desktop smoke test. No schema or migration files changed.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
| if (priorEpoch === undefined || nextEpoch === undefined || priorEpoch === nextEpoch) { | ||
| return false; | ||
| } | ||
| return priorEpoch > nextEpoch; |
There was a problem hiding this comment.
P2: runEpoch is an in-memory RuntimeKernel counter, so it restarts at 0 when the Host process restarts (runtime-kernel.ts:2202-2209), while this renderer-owned catalog survives the reconnect (desktop-feature-services.tsx:67-73) and the persisted catalog row can retain the same revision. This comparison then rejects every fresh same-revision row until the new Host's counter exceeds the old one. I reproduced the reducer path with an old row {revision: 5, runEpoch: 8, runningTurnIds: ['turn-1']} followed by the restarted Host's {revision: 5, runEpoch: 0, runningTurnIds: []}: commitSessions keeps the stale running row; patches at epochs 1–3 are also rejected. Please give the ordering token a Host generation/persistent scope, or reset the old comparison state on reconnect, and cover this restart sequence.
| ); | ||
| }); | ||
|
|
||
| test('decodes a Session attention payload on a catalog change', () => { |
There was a problem hiding this comment.
P3: This rebase removes main's decodeHostFrame regression test for session.catalog.changed frames carrying an attention payload. The other Host change-feed tests assert publication, not decoding, and this run-epoch change does not replace that coverage. Please retain the unrelated decoder test when rebasing.
|
Both addressed on the rebased branch (head
One scope note on the P2 fix: seeding from the construction clock means the epoch's magnitude is time-based rather than a small counter — within a Host run it is still monotonic per session, and across restarts the clock ordering is what makes fresh reads win. If you'd prefer a persisted generation counter instead of the clock base, I'm happy to rework it. |
585a1e3 to
fe8e7d8
Compare
hqhq1025
left a comment
There was a problem hiding this comment.
Reviewed the 15-file change at commit fe8e7d86 (including the latest four-file follow-up). The new tests restore attention decoding and exercise a larger catalog epoch, but the restart ordering is still not guaranteed; see the inline P2.
[P2] Remove the unrelated j1.txt CI runner log. It adds 18,160 lines / 1.8 MB to this PR and introduces 580 git diff --check whitespace errors. Diagnostic logs belong outside the tracked tree.
Local Node 24 build:test and 52 focused runtime/protocol/desktop tests passed. No checks are currently reported for this head, and the PR conflicts with current main in packages/runtime-host/src/protocol/index.ts: both branches assign compatibility epoch 192 to different wire changes. I did not run the complete test suite or a real Host-restart/Desktop smoke test. No schema or migration changes were found.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
| // while Desktop's catalog rows survive Host restarts. Seeding every epoch | ||
| // with the construction clock keeps a restarted Host's epochs strictly | ||
| // greater than anything the previous process produced (#5713 review). | ||
| this.#sessionRunEpochBase = deps.now?.() ?? 0; |
There was a problem hiding this comment.
[P2] A wall-clock seed does not guarantee a newer epoch after a Host restart. If the clock moves backward (or the old process increments its counter more times than the milliseconds elapsed before restart), the new Host starts below the old row's epoch. The same-revision comparison in session-catalog-state.ts then rejects its fresh catalog reads, leaving stale runningTurnIds until the new counter catches up. With two compiled kernels seeded at 1001 and 1000, the restarted value is 1000, not greater than 1001. Please use a restart-aware ordering/invalidating mechanism rather than assuming now() is monotonic across processes, and test the actual old-Host/new-Host transition.
Starting or ending a Turn does not move the catalog revision, so two reads of one session at the same revision can carry different runningTurnIds with nothing to order them — Desktop kept whichever landed last, flipping a responding row back to idle until the next status patch (apache#5713). - RuntimeKernel bumps a per-session epoch when a run enters or leaves the active set, including through host-operation claims whose runs are visible via the claim; SessionManager projects it next to runningTurnIds. - SessionCatalogLiveRunState may carry that epoch as an optional runEpoch; the decoder accepts the two-field form from hosts that do not track it and validates the epoch as a non-negative safe integer. The protocol compatibility epoch moves to 192 (an epoch-191 peer rejects the unknown key). - Desktop's catalog isStaleSummary orders equal-revision rows by the epoch and falls back to today's behavior when either side lacks one. The observer union in the main-process list summary stays untouched. Carries the apache#5713 review findings (test compilation, coordination-run visibility transitions, and the epoch rebase). Fixes apache#5713 Generated-by: GLM-5.3-Flash (ZCode)
…state The two-client UDS suites run only on Linux, so the runEpoch field the seed now projects was visible in CI but not on a Windows checkout: the known-empty fixture lacked the field the projection adds. Generated-by: GLM-5.3-Flash (ZCode)
…attention decoder test Two review follow-ups: - The per-session run-epoch counters restart at zero with a fresh Host process, while Desktop's catalog rows survive the restart — the epoch comparison then rejected every fresh same-revision read until the new counter outgrew the old one, pinning a stale session row across a Host restart. Seed the epoch with the construction clock so a restarted Host's epochs are strictly greater than anything the previous process produced. - The epoch rebase dropped main's decodeHostFrame regression for session.catalog.changed frames carrying an attention payload; restore it, and extend the Desktop ordering regression with the restart scenario. Generated-by: GLM-5.3-Flash (ZCode)
j1.txt was a GitHub Actions log saved while triaging a CI failure and was accidentally included in the previous commit. Generated-by: GLM-5.3-Flash (ZCode)
…st generation Second round on the run epoch (apache#5713): a wall-clock seed does not buy cross-process ordering. If the clock moves backward between restarts, or the previous process bumps its counter more times than milliseconds elapse before the restart, the restarted Host seeds below the epoch already recorded on a Desktop catalog row and every fresh same-revision read is rejected as stale until the new counter catches up (apache#5713 review). Stop pretending the epoch is comparable across processes and make the generation explicit instead: - RuntimeKernel exposes `sessionHostGeneration()` — an identity fixed at construction — and the per-session epoch returns to a plain per-process counter. Within one process the counter keeps ordering same-revision reads; across processes it claims nothing. - The Session catalog live run state carries `hostGeneration` (wire-visible, so the compatibility epoch moves to 194). Clients compare generations first: same generation orders by epoch, different generations never order — a restarted Host takes the row over from its predecessor whatever the two counters read, because every observation the predecessor published describes turns that no longer exist. A patch that lagged behind a restart survives at most until the live generation's next read. - Desktop's catalog staleness fence implements that comparison; the same-generation epoch protection from apache#5713 is unchanged. Verification: kernel interaction, session catalog protocol, coordinator and authenticated WebSocket suites pass locally; the Desktop suite pins the old-Host/new-Host transition (a fresh generation at a lower epoch must win, same-generation ordering must keep rejecting older reads). The session-catalog restart test now also asserts the generation changes across a service restart. The two-client UDS suite cannot run on Windows; its fixtures were updated to match and left to CI. Generated-by: GLM-5.3-Flash (ZCode)
fe8e7d8 to
c0b34f2
Compare
|
Fair point — a wall clock is not monotonic across processes, so the seed could not guarantee ordering. Replaced on the rebased branch (head
The two-client UDS suite cannot run on this Windows machine; its fixtures were updated to match and left to CI. |
The construction-time generation call consumed the first `newId` — shifting every id the kernel handed out against fixtures that pin the sequence, and crashing deps that omit it. Give the generation its own optional dep and default to a random UUID instead. Generated-by: GLM-5.3-Flash (ZCode)
|
Heads-up while CI reruns: the two red rounds after the fixes were both |
hqhq1025
left a comment
There was a problem hiding this comment.
Reviewed current head b76062d736ee73cb483d4845b97072bcabb4a772. I found one P2 correctness issue in the renderer's cross-Host ordering; see the inline comment.
The change adds a per-session run epoch and a random Host-generation identity, projects both through the catalog protocol, and rejects older same-generation observations at equal Session revision. The former clock-based ordering concern is removed, the stray CI log is gone, and the attention decoder test is restored. The current-head test check passed; the diff check and merge-tree against current main are clean. There is no storage schema migration.
I did not run the suite locally (this checkout has Node 18 and no dependencies) or reproduce the restart race in a packaged Desktop. The regression test covers new-Host takeover and older reads within that Host, but not an old-Host response arriving after takeover. This comment is not merge approval.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
| nextGeneration !== undefined && | ||
| priorGeneration !== nextGeneration | ||
| ) { | ||
| return false; |
There was a problem hiding this comment.
P2: A different random generation is treated as newer in both directions. If a read from Host A is delayed, Host B restarts and its same-revision row is committed, then A's already-issued response arrives, this branch returns false and commitPatch/commitSessions replaces B's row with A's stale runningTurnIds. The old row can remain until another catalog read/event. The existing test checks A→B and an older B epoch, but not B→late A. Generation identity alone cannot establish chronology; fence outstanding reads when the Host changes or carry an authoritative ordered incarnation, and add the reverse-arrival regression.
…e wire Connection close rejects every in-flight request with connection_lost (client/connection.ts), so a lagging predecessor read never delivers after the restarted Host's row has landed — the reverse-arrival case the review asked about needs a success across a dead connection, which the transport cannot produce. Generated-by: GLM-5.3-Flash (ZCode)
|
Reviewed the mechanism and I believe this exact arrival order cannot be produced by the transport. Closing a connection rejects every in-flight request with |
hqhq1025
left a comment
There was a problem hiding this comment.
I re-reviewed the exact head 2efb1580b12ee74aabef99493e748b6314f63eb1 (14 PR files, +538/-60). The only delta from the previously reviewed b76062d7 is the explanatory comment in session-catalog-state.ts:198-205; runtime behavior is unchanged. I revisited my earlier P2 about a predecessor Host row arriving after a successor row. The pure isStaleSummary function would accept such an ordering, but I could not establish that ordering through the production read path: connection failure rejects pending Host requests (packages/runtime-host/src/client/connection.ts:1021-1044), the Main IPC router retries a reconnectable read when its handler was replaced (apps/desktop/src/main/runtime-host-reconnecting-ipc-main.ts:309-324), and the renderer serializes full-list refreshes and same-session patch reads (session-read-state.ts:40-59, session-catalog-sync.ts:39-55). I therefore do not carry the earlier P2 forward as a substantiated current-head defect. I found no other substantiated P0-P3 issue in this re-review.
The exact-head test check succeeded. The PR diff passes git diff --check and a synthetic merge against fresh main (18827d99) is clean; GitHub currently reports MERGEABLE/BLOCKED, not a merge approval. I did not run the suite locally (Node 18/no installed dependencies), a packaged Desktop Host restart, or an Electron IPC-ordering probe for a response already settled before disconnect. Those remain validation gaps. No storage schema migration is involved.
Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.
Astro-Han
left a comment
There was a problem hiding this comment.
Second independent review lineage at 2efb1580 (supplements the earlier review on this head).
No P0–P2. I traced every path by which an old-Host row could reach the renderer catalog after a new-Host row at the same revision, and none can deliver an authoritative predecessor row that still carries its running state:
sessions:getgoes through the pending-request rejection on disconnect (connection.ts) and the handler re-check inruntime-host-reconnecting-ipc-main.ts, so a predecessor reply is always sent before any successor reply.- A predecessor list result still pending in preload is discarded, because any successor row admitted via
getmarks the refresher dirty (runtime-host-session-catalog.ts,preload.ts). sessions:changedcarries only an id, so it can only trigger a read that routes to the new Host.
The comparator, the kernel epoch increments (reserve/unregister/claim attach/settle), the protocol decoding, and the Desktop field pass-through all look correct. The new renderer and kernel tests would fail on the old code.
Non-blocking P3s:
- P3 — the comment overstates the guarantee (inline). One cross-generation row can arrive late: the main-process cache (
session-local-service.tscatalog()) returns predecessor rows aslocalState: 'cached', withrunningTurnIdsstripped butrunEpoch/runHostGenerationkept, until the new Host's catalog refresh completes. Because different generations are never stale, such a row can briefly overwrite the successor's running row. This matchesmain(same-revision last-writer-wins), so it is not a regression. - P3 — same-generation reconnect no longer degrades to cached. If the Host process survives and only the connection drops, a cached row carries the epoch from the last catalog refresh. If turns started or stopped since then, the renderer's newer epoch makes the cached row stale, so the row keeps showing the pre-disconnect running state instead of degrading as before. It recovers on reconnect. This is arguably more accurate, but worth confirming it's intended given what
localState: 'cached'is meant to signal. - P3 — minor. A third per-process identity is added alongside the existing
hostEpoch. Shared/guest session projections don't carry the new fields, so the #5713 flip remains for shared sessions. There's no regression test for the host-operation claim attach/settle increments.
Not verified: tests were not run locally (static read).
This review was produced with automated assistance (AI review agents) and checked by a maintainer-side reviewer before posting.
| * cross-generation response cannot exist on the wire, either: closing a | ||
| * connection rejects every in-flight request with `connection_lost` | ||
| * (client/connection.ts), so a lagging predecessor read never delivers after | ||
| * the successor's row has landed. |
There was a problem hiding this comment.
P3 — This holds for authoritative Host reads, but not for every row that reaches the catalog. After a restart, session-local-service.ts catalog() can still return the predecessor's rows as localState: 'cached' (runningTurnIds stripped, but runEpoch/runHostGeneration kept) until the new Host's catalog refresh lands. Since cross-generation rows are never stale, such a row can briefly overwrite the successor's row. That matches the old last-writer-wins behaviour, so it's not a regression. Suggest narrowing the wording to "an authoritative (non-cached) predecessor read never delivers late", and optionally not emitting runEpoch/runHostGeneration on cached rows.
Astro-Han
left a comment
There was a problem hiding this comment.
Approved at @Astro-Han's explicit request: independent automated reviews of this head from two model families found no blocking (P0–P2) issues, and CI is green. The remaining P3 notes (comment wording, cached-row behaviour) are non-blocking follow-ups.
Fixes #5713
Summary
revision, so two reads of one session at the same revision can carry differentrunningTurnIdswith nothing to order them. Desktop kept whichever landed last, so a read taken before the turn started — but landing after the running patch — flipped a responding row back to idle until the next status patch.RuntimeKernelnow bumps a per-session epoch every time a run enters or leaves the active set (turn start and end), andSessionManagerprojects it next torunningTurnIds.SessionCatalogLiveRunStatemay carry that epoch as an optionalrunEpoch. The decoder accepts the two-field form from hosts that do not track it (subset decoding, not exact-keys) and validates the epoch as a non-negative safe integer; the protocol compatibility epoch moves to 190 because an epoch-189 peer rejects the unknown key.isStaleSummaryorders equal-revision rows by the epoch (higher wins) and falls back to today's behavior when either side lacks one. The observer union in the main-process list summary is deliberately untouched — whether it is redundant under the ordered projection is a separate call.Verification
runEpochdecodes; a two-field live run state stays two fields; negative and fractional epochs rejectsession-catalog-coordinatorsuitesession-catalog-protocolsuitebiome formaton the formatted-path filesNot verified locally: full CI on Linux, and the real-app timing repro — the regression pins the ordering rule the fix introduces.
AI use
Implemented with GLM-5.3-Flash (ZCode): the counter, the wire field, the catalog ordering, and the tests at each layer.