Skip to content

feat(record): capture Trace v3 states after pages settle - #103

Open
shnpd wants to merge 3 commits into
feat/trace-v3-protocolfrom
feat/record-settled-states
Open

feat(record): capture Trace v3 states after pages settle#103
shnpd wants to merge 3 commits into
feat/trace-v3-protocolfrom
feat/record-settled-states

Conversation

@shnpd

@shnpd shnpd commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

背景

本 PR 基于 feat/trace-v3-protocol(PR #101),在扩展里落地 Trace v3 录制侧 observation 库,不改协议 wire / handshake / schema。

覆盖:

  1. 按 document settle:主文档、same-process iframe、OOPIF 各自探测 DOM 安静
  2. record-safe VOM 捕获 + state 去重:不带回原始 AX/DOM
  3. 统一几何匹配:顶层视口 CSS 像素,把操作目标对齐到 VOM ref
  4. Settle 队列 + redirect 合并:保序、可取消、iframe 可指定 scope
  5. Observation 序列化# bsk-observation 1 + steps_here + 行内注解
  6. Trace v3 reducer / buildTraceV3
  7. 拆出 v2 reducerrecord.ts 默认仍走 buildTraceV2

尚未把线上 record 接到 v3。结束录制仍出 Trace v2。本改动先把管线搭完(settle → observe → 几何匹配 → reduce → buildTraceV3),接线留给后续。

相对旧稿:文件收到 lib/recording/;几何不再做 documentRect / 滚动补偿,改为统一顶层视口;观察走已有 record-safe DTO。

动机

固定延时既跟不上一帧内的弹窗,也等不住慢路由。改为:页面(或目标 iframe)DOM 安静后再拍 observation,并用操作瞬间的几何 / 语义把目标对齐到该观察里的控件。Trace v3 因此同时描述「做了什么」和「在哪个控件、哪个页面状态上做的」。

Trace v3 形态(类型已在 #101):

  • states[] — 去重后的页面观察(s1s2、…)
  • steps[] — 每步绑定 state(操作前)与 result.state(操作后);目标优先带 VOM ref,对不上则保留 capture 的 role/name 并标 unmatched

架构 / 数据流

用户操作
  content script:CaptureTargetDescriptor(role / name / tag …)
  draft:captureTarget + 可选 targetHint { geometry, frameId }
        │
        ▼
 waitForDocumentSettled(scope)       ← MutationObserver;iframe / OOPIF 走独立 world / CDP session
        │
        ▼
 captureRecordingObservation()       ← record-safe VOM;节流 ≥200ms;失败重试一次
        │
        ├─ RecordingStateRegistry    ← url + VOM body 去重;s1, s2, …
        ├─ matchObservationTarget()
        │     frameId(缺省 = root,不跨 frame 扫)
        │     + tag + 顶层 rect ±2px
        │     几何唯一 → ref
        │     几何多命中 → role/name 收到 1 个
        │     无几何 → 仅在该 frame 内用 role/name
        │     否则 unmatched,保留 capture 语义
        ├─ bindDraft
        │     有观察:写 preStateId、steps_here、行内注解
        │     无观察:只写 unmatched target,不写空 state
        │     上一动作仍在 settle 且本次 unmatched:不绑陈旧 origin
        └─ SettleController
              schedule:串行、可取消;取消时用下一拍 origin 补前一步 postStateId
              scheduleRedirect:中间跳不落库;过期 hop 读到旧 URL 也不覆盖新 hop
              settleTrailing:stop 时给未闭合 draft 补同一拍 post-state
                │
                ▼
         draft.preStateId / postStateId / matchedTarget 就绪
                │
                ▼
         inferMissingPostStates()    ← 仍缺 post 时用下一步 pre
         reduceTraceStepsV3()
         buildTraceV3()              ← 只发布被引用的 states,重编号 s1..sN
                │
                ▼
         TraceV3 { version, entry, recorder, states, steps }

@shnpd
shnpd requested review from Ljy-0827 and iuyo5678 and a lite review from Copilot August 18, 2026 06:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR lays down the Trace v3 recording-side observation pipeline in the extension (page settle → capture VOM observation → target geometry matching → v3 reduction → TraceV3 build), while keeping the current default recorder export on Trace v2 by splitting out the v2 reducer.

Changes:

  • Add page “settle” detection via an injected MutationObserver probe (waitForPageSettled) and centralize recording constants (settle budgets, hashing, id generation, defaults).
  • Introduce observation/state registry + target matching utilities (record-observation.ts, match-target.ts, format-observation-file.ts) to capture and dedupe states and attach step annotations.
  • Split Trace v2 reduction into trace-reducer-v2.ts and switch record.ts to buildTraceV2, while evolving the existing trace-reducer.ts into a Trace v3 reducer that emits state-linked steps.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
apps/extension/src/transport/types.ts Expand draft step payloads with v3 fields (state ids, geometry, captureTarget) and add v3 types.
apps/extension/src/tools/vom/capture.ts Add documentRect and derive viewport localRect from document coordinates for robust matching after scroll.
apps/extension/src/tools/vom/tests/capture.test.ts Update tests to validate documentRect preservation and derived viewport rects.
apps/extension/src/tools/record.ts Keep default recording output on v2 by switching to buildTraceV2.
apps/extension/src/lib/trace-reducer.ts Implement v3 reducer producing states[] + steps[] with state/result references and redirect collapsing semantics.
apps/extension/src/lib/trace-reducer-v2.ts New v2-only reducer extracted to preserve legacy output behavior.
apps/extension/src/lib/record-observation.ts New observation pipeline: throttled capture, state dedupe, settle queueing, redirect coalescing, annotation + TraceV3 builder.
apps/extension/src/lib/record-constants.ts New centralized constants + FNV-1a hash + state id generator + navigation cause mapping defaults.
apps/extension/src/lib/page-settled.ts New “settle” implementation based on DOM quiet probing and readyState polling.
apps/extension/src/lib/match-target.ts New geometry-based matching from content-script capture to VOM refs (viewport vs document coordinates).
apps/extension/src/lib/format-observation-file.ts New observation file serialization (front matter + inline step annotations).
apps/extension/src/lib/describe-target.ts Rename content-script descriptor to CaptureTargetDescriptor (keep deprecated alias).
apps/extension/src/lib/tests/trace-reducer.test.ts Update tests to validate v3 reducer semantics (states, redirects, step id mapping).
apps/extension/src/lib/tests/trace-reducer-v2.test.ts New tests to validate extracted v2 reducer behavior.
apps/extension/src/lib/tests/record-observation.test.ts New tests covering annotation redaction behavior, target matching fallback, redirect coalescing.
apps/extension/src/lib/tests/page-settled.test.ts New tests for settle floor and cancellation behavior.
apps/extension/src/lib/tests/match-target.test.ts New tests for geometry matching and fallback/unmatched rules.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread apps/extension/src/lib/page-settled.ts Outdated
Comment thread apps/extension/src/lib/format-observation-file.ts Outdated
@shnpd
shnpd requested a lite review from Copilot August 18, 2026 07:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shnpd
shnpd force-pushed the feat/record-settled-states branch from e7a4b48 to 1afd217 Compare August 18, 2026 07:36
@shnpd
shnpd marked this pull request as draft August 18, 2026 08:59
@Ljy-0827
Ljy-0827 force-pushed the feat/record-settled-states branch from 1afd217 to 463a197 Compare August 19, 2026 05:02
@Ljy-0827
Ljy-0827 force-pushed the feat/record-settled-states branch from 463a197 to b2b5a35 Compare August 19, 2026 07:05
@Ljy-0827
Ljy-0827 marked this pull request as ready for review August 19, 2026 07:12
@Ljy-0827
Ljy-0827 force-pushed the feat/record-settled-states branch from b2b5a35 to c58b6bf Compare August 21, 2026 03:46
@iuyo5678
iuyo5678 force-pushed the feat/record-settled-states branch from c58b6bf to a93b9ef Compare August 21, 2026 08:21
@shnpd
shnpd force-pushed the feat/record-settled-states branch from a93b9ef to 76f6867 Compare August 21, 2026 10:27
@shnpd
shnpd force-pushed the feat/record-settled-states branch from 76f6867 to 2853ff1 Compare August 22, 2026 09:01
shnpd and others added 3 commits August 23, 2026 21:26
Wait for DOM quiet after actions, match targets to VOM refs, and reduce
drafts into deduped state-linked Trace v3 while keeping the v2 reducer.

Co-authored-by: Cursor <cursoragent@cursor.com>
@iuyo5678
iuyo5678 force-pushed the feat/record-settled-states branch from 2853ff1 to 6300eda Compare August 23, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants