Protect the source authority register behind the RAG impact gate, and log the red canary - #2732
Open
claude[bot] wants to merge 1 commit into
Open
Protect the source authority register behind the RAG impact gate, and log the red canary#2732claude[bot] wants to merge 1 commit into
claude[bot] wants to merge 1 commit into
Conversation
… log the red canary Two pieces of RAG-safeguard hygiene found while preparing PR #2711. pr-policy classified `src/lib/source-authority-registry.ts` as an ordinary file, so PR #2711 declared its RAG impact voluntarily rather than because the gate asked. That file and `src/lib/australian-source-priority.ts` do sit on the ordering path: classifySourceAuthority assigns a result's tier, australianSourceTier reads it, and selectAustralianClinicalContext orders and trims the model's context by it. Registering or re-scoping a publisher moves that ordering without touching a score, which is exactly the shape the gate exists to catch. Both are now in ragRankingPatterns, asserted in the self-test, and listed in the protected-surface table. This is authority classification, not a governance ranking penalty or boost. That shape stays refuted and the safeguards note says so, so a later reader cannot mistake this entry for permission to revisit Refutation 3. The second piece is an outstanding-issues inbox request. The eval canary has been red on the same three answer-quality cases since at least 2026-08-23, over five consecutive runs, while golden retrieval passed 36/36 in every one. Nothing in the ledger tracked it. A permanently red canary is a poor instrument to validate the next protected change against. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DXnbx8Pa4VMp2dzBrYFH4J
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/pr-policy.mjsclassifiedsrc/lib/source-authority-registry.tsas an ordinary file, so Register nine Australian clinical publishers in the source authority register #2711 carried itsRAG impact:line voluntarily rather than because the check demanded it. That file andsrc/lib/australian-source-priority.tsare on the ordering path:classifySourceAuthorityassigns a result's tier,australianSourceTierreads it, andselectAustralianClinicalContextorders and trims the model's context by it. Registering or re-scoping a publisher moves that ordering without touching a score, which is precisely the shape this gate exists to catch. Both are now inragRankingPatterns, asserted in the self-test, and listed in the protected-surface table indocs/rag-behaviour/safeguards.md.refuted-approaches.md§ Refutation 3, ledger#032), and saying so here stops a later reader mistaking this entry for permission to revisit it.clozapine-monitoring,clozapine-monitoring-paraphrase,nocc-requirements,source_backed_review_fallback unaccounted), going back to run32656925301on 2026-08-23. The request records the evidence, points at#J8SJQ9as the overlapping row to check first, and explicitly forbids the two tempting non-fixes: addingacceptSourceOnlyto the eval cases, or relaxing the threshold.Filed as an inbox request, not a direct ledger-table edit, per the write-discipline rule. Run
npm run issues:reconcilefrom a dedicated ledger branch after this lands.Why this is not just paperwork
The canary is the pair-comparison instrument for every protected RAG change, including the one #2711 is about to be measured with. A permanently red run trains readers to skim past it, and a real retrieval regression would then land inside an already-red signal.
Verification
npm run check:pr-policy— self-test passed, workflow guard passed. The two new assertions fail against the old pattern list, so the change is red-proven.npm run check:ledger-write-discipline— passed for0177bed18446..HEAD. This gate compares committed refs, so it was re-run after the commit, not before.npm run check:outstanding-issues— snapshot in step, 121 open, 0 pending.npm run docs:check-links— 7392 repo path references resolve.npx prettier --check .— whole tree clean, formatting committed.npm run verify:ui— not applicable, no UI, routing or styling change.npm run eval:retrieval:quality— Verification not run: not applicable. No retrieval, ranking, selection or scoring code path changes here. This PR only changes which files the PR-body gate classifies, plus documentation and one ledger request.Risk and rollout
RAG impact:line. It cannot change retrieval, ranking, answers, or anything at runtime.Clinical Governance Preflight
Clinical KB Database(sjrfecxgysukkwxsowpy)This PR adds a guard and a tracking record. It removes no control and relaxes no threshold.
Notes
🤖 Generated with Claude Code
https://claude.ai/code/session_01DXnbx8Pa4VMp2dzBrYFH4J
Generated by Claude Code