Repository navigation
feat: Filter tool declarations and recheck eligibility before execution - #6143
markstuart-oai wants to merge 3 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
Security findingsAdvisory findings (1)
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4d1a13abde
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Request changes at 4d1a13abde47568fb17ac93d55ebb260e3aae32e. I independently confirmed the existing P2 sandbox inventory finding. prepare_sandbox_agent() adds capability FunctionTools such as ExecCommandTool to a cloned execution agent, while both run loops pass the public agent to prepare_declarations(). The new membership check therefore rejects these tools even with a pass-through declaration callback, before the model request. Please validate against the resolved execution inventory while preserving public agent identity for callbacks, and add regressions for SandboxAgent capability tools in both Runner modes, including approval/resume.
Reviewed the remaining declaration binding, forced-choice, stream isolation, MCP filtering and resume paths with an independent review; no additional actionable findings. Referencing the existing inline finding rather than duplicating it. Hosted CI is complete: 33 checks passed and 4 skipped; no failures. The sandbox inventory finding above remains unresolved. Static review only; no candidate code or local tests executed.
There was a problem hiding this comment.
🛡️ Codex Security Review · Automatically triggered
Here are some automated security review suggestions for this pull request.
Reviewed commit: 464021ccec
ℹ️ About Codex security reviews in GitHub
This is an experimental Codex feature. Security reviews are triggered when:
- You comment "@codex security review"
- A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review
Once complete, Codex will leave suggestions, or a comment if no findings are found.
There was a problem hiding this comment.
Superseded by request changes: confirmed mixed direct/programmatic caller policy gap.
Approved after static re-review and two independent reviews. The sandbox inventory finding is fixed: declaration preparation validates the execution inventory while preserving public agent identity for callbacks. Reviewed capability source tracking through preparation and caching, namespaced tool copy semantics, exact method/type/schema/approval rebinding, the SDK patch adapter, serialized capability-source digests, and direct-tool revocation. Pending approvals cannot move between direct tools and capabilities or change capability position/type/key. Regression coverage exercises streamed and nonstreamed execution, in-memory and serialized resume, replacement executors and approval policies, and source changes. No remaining actionable findings.
Hosted CI snapshot: 17 passed and 7 running; remaining checks require follow-up. Static review only; no candidate code or local tests executed.
There was a problem hiding this comment.
Request changes: supersedes my approval at this commit. I missed the mixed-caller gap identified in the existing P2 finding; independent static checks now confirm it. _covered excludes every function containing programmatic in allowed_callers, including functions also allowing direct calls. Such a tool remains advertised without invoking prepare_tool_declaration. A call with no caller metadata is accepted as direct, and the missing prepared binding skips the execution-time policy recheck. Cover mixed tools for direct invocation, or reject this configuration when declaration preparation is enabled, with a denial regression. Referencing the existing finding rather than duplicating it.
The sandbox inventory and approval/resume fixes remain sound. Hosted CI completed: 33 passed, 10 skipped. The mixed-caller correctness finding remains unresolved. Static review only; no candidate code or local tests executed.
There was a problem hiding this comment.
Approved after static re-review and an independent second review. The mixed-caller bypass identified in the previous review is fixed: enabling declaration preparation rejects mixed direct/programmatic tools, including live and restored approvals. Inventory validation prevents unprepared programmatic-only functions from gaining direct access during selection or request hooks. Reviewed caller-list detachment, both Runner modes, active empty-inventory versus disabled-policy handling, and the denial regressions. No new actionable findings.
Hosted CI completed: 33 passed, 8 skipped, no failures. The macOS build and downstream installed-wheel verification succeeded after the earlier Rust-download DNS failure. Static review only; no candidate code or local tests executed.
This pull request adds an experimental Runner policy for direct functions, sandbox capability functions and tools discovered through
agent.mcp_servers. For example, an application can withhold a publish tool until an inspection succeeds and revoke access while an approved call waits to run.Summary
Set
ToolExecutionConfig.prepare_tool_declarationto an async callback. It receives the run context and an immutableToolDeclaration. Return a description to offer the tool orNoneto withhold it. The selection governs both the model request and the calls the Runner accepts. The callback cannot replace the executor, schema or approval policy.The Runner rechecks eligibility before execution, including after queueing, approval and MCP metadata resolution. Withholding a named tool choice or leaving a required choice with no tools raises an error before the model request.
With this policy enabled, a function that allows both direct and programmatic callers raises
UserErrorduring selection. Programmatic-only functions stay outside the policy. Changing their caller list after selection cannot grant direct access for that request.For sandbox functions, the Runner checks the session's tool inventory and passes the public agent to
is_enabled. Removing a direct function from that agent blocks execution even if the sandbox still lists it. A pending capability approval records the capability type, position and tool key as a digest. An approval cannot move between a capability and a direct function with the same name.Keep the callback free of side effects and supply it again on resume. Restoring a serialized approval relies on the application's current authorized tools. The policy does not order parallel calls or cover provider-managed tools. It is off by default.
Test plan
view_image.on_llm_startand across live and serialized approvals.Checks
.agents/skills/code-change-verification/scripts/run.sh.