Catalogue structured calculator evidence sources - #2712
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_ac28d2eb-21e1-41c5-9085-2b7e6dbca8f7) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f60765a5c2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex resolve actionable Codex review findings for this pull request and current head using the repository instructions. This is the pull request's single automatic repair pass: do not perform a fresh review, create new standalone findings, or request another review. Work only the existing unresolved Codex threads on the current head. The only repository destination is BigSimmo/Database, and the only branch destination is the pull request head branch codex/fix-calculator-sources at starting commit f60765a; never publish fixes to a detached or synthetic work branch and never create a stacked pull request. Use the authenticated GitHub connector to commit each approved fix to BigSimmo/Database:codex/fix-calculator-sources, then verify that the pull request head contains the published commit before reporting success. Always fix P0 and P1 findings. For P2 and lower findings, fix only clear, scoped, low-risk issues; otherwise disposition them with a concise reason. For a fixed thread, reply with as the first line and as the second line. For a no-code disposition, use followed by . These result markers authorize the workflow to close that exact thread only after it verifies a fixed commit is the pull request head; a local-only commit is not a fix. If publication or verification fails, do not use either result marker, do not claim success, and leave the thread open with the blocker. If human input or new authorization is required, do the same. Finish only after every actionable thread is fixed or dispositioned and closed, or explicitly left open for a human decision. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation. |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_5fecc866-517d-41b1-9125-39932a3ab3a9) |
… not active The registry types status as an unrestricted string and the reader treated everything except two named values as active, so a missing, misspelled or newly introduced status would have presented ambiguous or quarantined evidence on /sources with no inactive or excluded warning. Two guards, because they catch it at different moments. The reader now maps status through an explicit table and falls closed to inactive for anything it does not recognise, so only "reviewed" can read as active. check:calculator-content enumerates the same three statuses and refuses the data outright, so introducing a status is a deliberate decision at the registry rather than a source silently demoted at every read. Tests cover both halves: the table admits exactly one active status, the fallback is inactive and not active, missing/misspelled/new statuses all resolve to inactive, and the shipped checker is run against a mutated copy of the real evidence file for each of those cases. 7 passed. check:calculator-content passes on the file as committed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0142trjgjRAP2GA9wzqoYUAE
…to fix/2712-source-status
2 participants
Summary
Verification
npm run verify:pr-localVerification not run: the focused Sources contract and typecheck passed locally; GitHub is the requested PR verification surface.
During development, use
npm run verify:cheapas the faster iteration gate before the final PR-local preflight.npm run verify:uiwhen UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changednpm run verify:releasebefore release or handoff confidence claimsUI verification not run: no UI, routing, styling, or browser behavior changed. Release verification was not requested.
For retrieval, ranking, selection, chunking, source/citation rendering, or answer-contract changes,
verify:pr-localrunseval:rag:offlineautomatically. Run the offline command directly during iteration before spending a live eval.npm run eval:retrieval:quality(must stay 36/36) when retrieval, ranking, selection, chunking, or scoring behavior changed — CI cannot run it (needs live keys), so run it locally and paste the summary. A metadata/governance-weighting change once buried correct docs (recall 1.0→0.76) and only this eval caught it.npm run eval:rag -- --limit 15+npm run eval:quality -- --rag-onlywhen answer generation, the synthesis prompt, or answer post-processing changed (grounded-supported must not drop; citation-failure 0)npm run check:production-readinesswhen clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changednpm run check:deployment-readinesswhen deployment startup, hosting, or rollout behavior changedLive retrieval and RAG evaluations were not run because retrieval, ranking, answer generation, and RAG behavior are unchanged. Production-readiness was not run because it requires separately authorized live/provider evidence. Deployment-readiness was not run because deployment behavior is unchanged.
Focused local evidence:
node scripts/run-vitest.mjs run tests/source-catalogue-core.test.ts tests/source-url-policy.test.ts tests/source-catalogue-providers.test.ts— 79/79 tests passed.npm run check:source-catalogue— passed; 8 calculator references, 1,639 captured references, and 775 canonical sources.npm run typecheck— passed.npm exec -- prettier --check src/lib/sources/repository-providers.ts src/lib/sources/source-url-policy.ts tests/source-catalogue-providers.test.ts— passed.git diff --check— passed before commit.Risk and rollout
f60765a5c2.Clinical Governance Preflight
Clinical KB Database(sjrfecxgysukkwxsowpy)Notes
Note
Low Risk
Changes affect source-catalogue metadata and governance checks only, not calculator scoring or RAG; mapping mistakes could misrepresent review or lifecycle status in the Sources UI.
Overview
Calculator sources in the repository catalogue now come from the structured evidence registry instead of one free-text
sourcestring per calculator. Each fixture’ssourceIdsdrive eight academic evidence entries (title, URL, issuer, review dates, jurisdiction, evidence type, validation, lifecycle, supersession) with usage on fieldsourceIds. Internal governance and rights-only records stay out of that academic set.Coverage checks are tightened so missing evidence IDs, missing catalogue usage per calculator, and unused academic evidence all fail validation. The calculators provider also lists
data/calculators/evidence.jsonas an owner path.Governed URL allowlist gains
www.abs.gov.auso a new calculator canonical URL passes policy. Tests expect eight calculator references, assert full academic provenance (including inactive/excluded lifecycles), and compare governed hosts to emitted URLs without a fixed host count.Reviewed by Cursor Bugbot for commit 8ba5d54. Configure here.