Repository navigation
feat: capture Runner submissions and provider request attempts - #6140
markstuart-oai wants to merge 2 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79564c8739
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
jbeckwith-oai
left a comment
There was a problem hiding this comment.
Request changes: independently confirmed the existing P2 guardrail accounting finding. In streamed runs, model_run_context_stream enters _model_submission before get_model_stream checks the input-guardrail tripwire. If the guardrail trips during awaited preparation, _StreamAttemptSkipped prevents model.stream_response from being called, yet the capture records one submission. Move registration after that gate while retaining one logical submission across retries, and add a regression for a guardrail tripping during input-filter or on_llm_start preparation. Referencing the existing comment rather than adding a duplicate inline finding.
Completed comprehensive static review with two independent reviews covering capture ownership, attached-child isolation, retries, automatic compaction, HTTP authentication/redirects and trace gaps, WebSocket send/reply boundaries, frozen snapshots, and disabled-capture behavior. No additional actionable findings.
Hosted CI snapshot: {"SKIPPED": 2, "SUCCESS": 17, "IN_PROGRESS": 7}. Remaining checks stay pending. Static review only; no candidate code or local tests executed.
There was a problem hiding this comment.
Approved after static re-review and an independent second review. The guardrail accounting finding is fixed: streamed submissions are registered only after the input-guardrail skip gate, in the task that actually opens the model stream. The per-stream flag preserves one logical submission across retries, and the capture scope restores the enclosing context. Reviewed timed and untimed retry paths, nonstreaming scope preservation, and the enabled/disabled capture regression matrix. No remaining actionable findings in the reviewed changes.
Hosted CI is complete: 33 checks passed and 4 skipped; no failures. Static review only; no candidate code or local tests executed.
This pull request adds
RequestCaptureto the Python Runner. One model submission can trigger several requests through retries, redirects or authentication. Callers can now inspect local transport counts even when a run fails or is cancelled.One capture covers the first Runner invocation in the block. Drain streams inside the block. Reusing the capture starts fresh, including after approval. Handoffs and corrections stay with the same owner. Attached agent tools share child totals. Automatic session compactions count separately.
observedisNone.Capture is off by default. Snapshots contain fixed labels and counts. The SDK creates a private client subclass to associate retries with their request while preserving the default User-Agent. Caller-supplied clients remain unchanged.
Verification
Formatting, lint, Mypy and Pyright passed. The full local suite passed 13,254 tests and skipped 37. All 35 capture tests also passed on Python 3.10 and with the minimum supported client version. The guardrail regression covers timed and untimed streams with capture enabled and disabled.