Repository navigation
feat(dependencies)!: assess advisories through the SDK and CLI - #1136
mldangelo-oai wants to merge 235 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
Security findingsBlocking findings (5)
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
soyeon-oai
left a comment
There was a problem hiding this comment.
Withholding approval pending the newly added native cancellation test failure below. Reviewed the runtime modules, API integration, input/package/advisory handling, provenance and partial-result/cancellation paths, schemas and key tests. No confirmed additional production defect from this source pass. No independent execution; ecosystem correctness and all 134 changed files have not been exhaustively validated.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 87913c05d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Broad cross-ecosystem process execution and model orchestration require final human review, and one misleading dependency-output error remains.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
Adds SDK-level software composition analysis using an installed OSV-Scanner, followed by optional static advisory triage.
Changes:
- Adds dependency discovery, OSV normalization, durable artifacts, assessment, reporting, comparison, and update handoff APIs.
- Introduces versioned SCA schemas, public types, documentation, and examples.
- Adds extensive ecosystem, process-boundary, failure-retention, and evaluation coverage.
| File | Description |
|---|---|
sdk/typescript/tests-ts/sca-triage.test.ts |
Tests triage contracts and prompts. |
sdk/typescript/tests-ts/sca-process-boundary.test.ts |
Tests executable and environment isolation. |
sdk/typescript/tests-ts/sca-invocations.test.ts |
Tests per-file invocation retention. |
sdk/typescript/tests-ts/sca-git-scope.test.ts |
Tests Git-scoped discovery. |
sdk/typescript/tests-ts/sca-exclusions.test.ts |
Tests exclusion handling. |
sdk/typescript/tests-ts/sca-ecosystems.test.ts |
Tests cross-ecosystem semantics. |
sdk/typescript/src/thread-source.ts |
Adds dependency-triage thread source. |
sdk/typescript/src/targets.ts |
Exposes configurable Git isolation. |
sdk/typescript/src/sca.ts |
Builds and persists SCA results. |
sdk/typescript/src/sca-types.ts |
Defines generated SCA contracts. |
sdk/typescript/src/sca-triage.ts |
Implements model triage contract. |
sdk/typescript/src/runtime.ts |
Adds persistent dependency outputs. |
sdk/typescript/src/index.ts |
Exports the public SCA API. |
sdk/typescript/src/api.ts |
Implements scanDependencies. |
sdk/typescript/scripts/sca-osv-ecosystem-fixtures.mts |
Defines ecosystem scanner fixtures. |
sdk/typescript/scripts/generate-sca-models.cjs |
Generates SCA TypeScript models. |
sdk/typescript/scripts/check-package.mjs |
Verifies packaged SCA modules. |
sdk/typescript/README.md |
Documents SCA usage and limitations. |
sdk/typescript/pnpm-lock.yaml |
Promotes saxes to runtime. |
sdk/typescript/package.json |
Adds generation and runtime dependency changes. |
plugins/codex-security/skills/triage-finding/references/triage-result-contract.md |
Documents the shared schema. |
plugins/codex-security/schemas/triage-result.schema.json |
Promotes the triage schema. |
plugins/codex-security/schemas/sca-result.schema.json |
Defines persisted SCA results. |
plugins/codex-security/plugin-files.json |
Packages the new schemas. |
examples/sca/run.mjs |
Adds a runnable scan example. |
examples/sca/README.md |
Documents SCA examples. |
examples/sca/handoff.mjs |
Demonstrates update handoffs. |
examples/sca/compare.mjs |
Demonstrates result comparison. |
examples/sca/ci.yml |
Shows CI artifact integration. |
evals/triage-finding/sca/scripts/sca-result.js |
Scores SCA evaluations. |
evals/triage-finding/sca/scripts/run-promptfoo.js |
Stages isolated evaluation runs. |
evals/triage-finding/sca/scripts/generate-tests.js |
Generates synthetic evaluation cases. |
evals/triage-finding/sca/scripts/baselines.js |
Produces scanner baselines. |
evals/triage-finding/sca/README.md |
Documents evaluation methodology. |
evals/triage-finding/sca/QA.md |
Records evaluation QA. |
evals/triage-finding/sca/prompts/triage-baseline.txt |
Defines the baseline prompt. |
evals/triage-finding/sca/prompts/sca-wrapper.txt |
Defines the typed-match prompt. |
evals/triage-finding/sca/promptfooconfig.yaml |
Configures model evaluation. |
evals/triage-finding/sca/fixtures/sca-001/src/application.mjs |
Adds affected depth-parser usage. |
evals/triage-finding/sca/fixtures/sca-001/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-001/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-001/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-001/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-001/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-002/src/application.mjs |
Adds mitigated depth-parser usage. |
evals/triage-finding/sca/fixtures/sca-002/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-002/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-002/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-002/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-002/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-003/src/application.mjs |
Adds unresolved depth-parser usage. |
evals/triage-finding/sca/fixtures/sca-003/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-003/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-003/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-003/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-003/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-004/src/application.mjs |
Adds affected template usage. |
evals/triage-finding/sca/fixtures/sca-004/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-004/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-004/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-004/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-004/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-005/src/application.mjs |
Adds mitigated template usage. |
evals/triage-finding/sca/fixtures/sca-005/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-005/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-005/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-005/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-005/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-006/src/application.mjs |
Adds unresolved template usage. |
evals/triage-finding/sca/fixtures/sca-006/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-006/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-006/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-006/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-006/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-007/src/application.mjs |
Adds affected cache usage. |
evals/triage-finding/sca/fixtures/sca-007/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-007/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-007/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-007/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-007/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-008/src/application.mjs |
Adds mitigated cache usage. |
evals/triage-finding/sca/fixtures/sca-008/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-008/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-008/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-008/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-008/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-009/src/application.mjs |
Adds unresolved cache usage. |
evals/triage-finding/sca/fixtures/sca-009/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-009/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-009/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-009/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-009/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-010/src/application.mjs |
Adds affected filter usage. |
evals/triage-finding/sca/fixtures/sca-010/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-010/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-010/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-010/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-010/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-011/src/application.mjs |
Adds mitigated filter usage. |
evals/triage-finding/sca/fixtures/sca-011/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-011/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-011/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-011/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-011/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/fixtures/sca-012/src/application.mjs |
Adds unresolved filter usage. |
evals/triage-finding/sca/fixtures/sca-012/package.json |
Declares fixture dependency. |
evals/triage-finding/sca/fixtures/sca-012/package-lock.json |
Locks fixture dependency. |
evals/triage-finding/sca/fixtures/sca-012/osv.json |
Captures synthetic advisory data. |
evals/triage-finding/sca/fixtures/sca-012/input.json |
Defines typed advisory input. |
evals/triage-finding/sca/fixtures/sca-012/APPLICATION.md |
Defines fixture deployment context. |
evals/triage-finding/sca/corpus/manifest.json |
Plans human-adjudicated corpus work. |
evals/triage-finding/sca/corpus/developer-pilot.csv |
Adds pilot recording template. |
evals/triage-finding/sca/corpus/adjudication-template.json |
Adds adjudication template. |
evals/triage-finding/sca/assertions/sca-metrics.js |
Records evaluation metrics. |
evals/triage-finding/sca/assertions/sca-evidence.js |
Validates verdicts and citations. |
evals/triage-finding/sastbench/promptfooconfig.sastbench.yaml |
Uses the promoted schema. |
evals/triage-finding/README.md |
Updates schema documentation. |
evals/triage-finding/promptfooconfig.yaml |
Uses the promoted schema. |
evals/triage-finding/promptfooconfig.calibration.yaml |
Uses the promoted schema. |
evals/triage-finding/promptfooconfig.calibration-smoke.yaml |
Uses the promoted schema. |
evals/triage-finding/package.json |
Adds deterministic and model SCA scripts. |
docs/sca-qa.md |
Documents implementation verification and risks. |
Files not reviewed (13)
- evals/triage-finding/sca/fixtures/sca-001/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-002/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-003/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-004/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-005/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-006/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-007/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-008/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-009/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-010/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-011/package-lock.json: Generated file
- evals/triage-finding/sca/fixtures/sca-012/package-lock.json: Generated file
- sdk/typescript/pnpm-lock.yaml: Generated file
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 24763698a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
alandelong-oai
left a comment
There was a problem hiding this comment.
Thanks—the Pipenv fixture now declares its public source, and the maintained OSV-Scanner 2.6.0 harness passes all 114 cases. This owner-level merge-safety review of 09b02507 found one additional issue in the full dependency-command contribution: a symlink followed by .. can select a different repository, as detailed inline. The fixture-only follow-up is sound.
Fresh validation passed 726 SCA/CLI tests (one Windows-only skip), 38 deterministic evals, 41 Deep Scan leaf cases, and nine portable tests; builds, types, formatting, and built CLI checks passed. The path issue was reproduced through source and built CLI modules with real filesystem paths and the SDK call intercepted before scanning. No full SDK suite, live model/provider assessment, or Windows execution is claimed.
alandelong-oai
left a comment
There was a problem hiding this comment.
Thanks for the fix. This owner-level merge-safety review of 7eb287dd found no material unresolved issue. Dependency scans now preserve physical symlink/.. repository selection through the shared path helper. The documented breaking scan dependencies selector remains intentional.
Fresh validation passed 726 dependency/CLI tests (one Windows-only skip), 114 offline OSV contract cases, 38 deterministic evaluations, 41 Deep Scan leaf cases, and nine source-checker tests. Builds, types, formatting, eight source/built Node CLI path controls, and eight independent resolver checks passed.
The broader shared path fixture remains blocked by local default-state-directory permissions after its dependency assertion. No full SDK suite, live advisory-service/model assessment, or native Windows/macOS execution is claimed.
alandelong-oai
left a comment
There was a problem hiding this comment.
Thanks for the thorough implementation. No blocking findings after an owner-level merge-safety review of all 156 changed files at 109c8e1, including the current main integration and resolved discussion. The intentional reservation of scan dependencies is documented; literal -- still selects a source path. Advisory facts remain separate from static assessments, with source identity, credential boundaries and partial-result checkpoints preserved.
Validation passed: 774 focused tests (one Windows-only skip), 228 CLI dispatch tests, 38 deterministic evals, 41 Deep Scan cases, 114 real OSV 2.6.0 offline contracts, eight source/built path checks, and a compiled CLI → real OSV → saved-artifacts check. Fresh builds, generated models, types, Ruff and formatting passed. The broader shared suite had 445 passes, 37 platform skips and three failures; all three reproduce on the exact base, and the two Python-path assertions pass on both commits with ordinary Python discovery.
Limits: no full suite, native Windows/macOS execution, live advisory/model run, or installed-package smoke. Broad eval type checking could not resolve the separate yaml/promptfoo dependencies. CI currently has 74 successful checks and two still running. Approving with those validation limits recorded.
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…ename variants together

Summary
Add local dependency assessment to the CLI with
cs scan dependencies [repository]and to the SDK withscanDependencies(). Operators can inspect the evaluated inputs, the original OSV advisory matches, and an assessment of each match against local source. Main’s scanner import accepts existing findings. This command performs matching itself.Changes
Testing
e93481d3, 775 affected SDK and CLI tests passed across 16 files, with one Windows-only case skipped. The 38 deterministic dependency evaluation tests and the Deep Scan worker settings test at the child-process boundary passed.e93481d3. The seeded run used12345and the default run selected2618295243. Each recorded 13,661 passes, 172 skips, one todo, zero failures and 148,231 assertions across 328 files.109c8e16recorded 13,429 passes, 173 skips and one todo in each of the two complete installed-package SDK test orders. These historical totals apply to that revision.Risk and rollout
Breaking CLI change. The bare argument in
cs scan dependenciesselects a dependency scan of the current directory. Usecs scan ./dependenciesfor a source scan of a directory nameddependencies.OSV-Scanner must be installed on PATH. The default output uses persistent dependency state. A completed scan returns 0 even when it finds advisories. Partial or failed scans and budget stops return 2, and interrupts retain 130 or 143. Cost limits use estimated spend.
Online matching sends package identities to OSV. Model assessment uses local source and can leave applicability unresolved. This refresh used synthetic inputs. Live matching and live model qualification remain outstanding. Go manifests with
toolchain defaultremain unsupported because the previously tested OSV-Scanner 2.6.0 also reads unselectedgo.sumentries in that case.Public disclosure review
Reviewed the public text, changes, commit messages, discussion, available linked logs and retained artifacts. Some cited CI runs have expired artifacts, so the comprehensive historical attestation remains open.