Repository navigation
feat: let agent hooks request another turn before accepting a final answer - #6124
markstuart-oai wants to merge 2 commits into
Conversation
…nswer Signed-off-by: markstuart-oai <323302876+markstuart-oai@users.noreply.github.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
Superseded by request changes: Python 3.10 cancellation assertion failures.
Approved after comprehensive static review and an independent second review. Reviewed text, structured and final-tool candidates; missing/empty content and refusal handling; approval/resume and handoff precedence; turn numbering and limits; provisional history; error-handler bypass; accepted-output guardrails and end hooks; and cancellation/error propagation. Completed tool effects are checkpointed before resumed policy evaluation and retained on policy failure, while parent cancellation joins the policy task without accepting a late finish. Existing behavior remains unchanged without a policy. No actionable findings.
Hosted check snapshot: {"SUCCESS": 14, "IN_PROGRESS": 9, "FAILURE": 1}. Pending checks require follow-up. Static review only; no candidate code or local tests executed.
There was a problem hiding this comment.
Request changes: supersedes my approval at this commit. I published that approval before inspecting the newly completed Python 3.10 failure. The hosted job reports six failures, all in new cancellation-message assertions: expected decision failed or local, actual empty string. Make these assertions compatible with supported Python 3.10 while retaining cancellation-type, checkpoint/session persistence and no-replay checks. No additional production correctness findings from the comprehensive static and independent reviews. Hosted CI is now terminal: 31 passed, 2 failed (the known Python 3.10 jobs), and 6 skipped. The request-changes decision remains in effect. No candidate code or local tests executed.
There was a problem hiding this comment.
Approved after static re-review and an independent second review. The Python 3.10 cancellation-test finding is fixed: only cancellation-message matching is relaxed on Python versions below 3.11, while RuntimeError messages, cancellation types, suppressed final hooks, exactly-once tool effects, resumed output, and persisted session results remain checked. The incremental change is limited to these tests; no remaining actionable findings.
Hosted CI is complete: 33 passed and 2 skipped; no failures. Static review only; no candidate code or local tests executed.
|
@codex review |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
This pull request adds an optional agent hook to reject premature answers. For example, one tool measures a width and another computes the area. The hook can require both results before the run ends.
Use
Agent(name="Surveyor", hooks=AgentHooks(step_policy=decide))to check the answer against application state. The callback receives the run context and aSettledStep. It can return"continue"to request another turn or"finish"to accept the answer. Both synchronous and asynchronous callbacks work in streamed and non-streamed runs.The hook runs once per proposed answer, after pending approvals resolve. Handoffs take precedence. Eligible answers include text, parsed structured output and tool results selected by
tool_use_behavior. Without the hook, existing behavior stays the same.Continuation skips end hooks and keeps the rejected answer in history. Stream events can include that provisional answer. Dynamic instructions can guide the next turn using the updated context. Turn limits still apply, and accepted answers still go through output guardrails.
Test plan
Checks
.agents/skills/code-change-verification/scripts/run.sh