ci: make Recommend integration tests workflow fork-safe#7861
ci: make Recommend integration tests workflow fork-safe#7861ianwinsemius wants to merge 1 commit into
Conversation
The Recommend integration tests workflow runs on every `pull_request` event, including PRs opened from forks. When triggered by a fork PR the provided GITHUB_TOKEN has read-only access to the upstream repo and any write call (addLabels / createComment) returns: HttpError: Resource not accessible by integration (HTTP 403) That failure surfaces as a red 'recommend' check on every fork PR even when the underlying source code is fine. It's also misleading: the job's job is to recommend integration tests, not to gate the PR on its own infra success. Fix: wrap the label and comment API calls in a small helper that catches 403s and logs an actionable message instead of throwing. The job runs to completion green, and maintainers can apply the 'integration-tests: recommended' label manually if the change warrants integration testing. Comment writes are additionally skipped entirely on fork PRs because the GitHub UI already shows a banner explaining how external contributors can request review — a bot comment to a fork PR author who cannot apply labels themselves would just add noise. This is a CI-only change with no runtime impact, so it does not include a changeset (per .github/skills/changesets/SKILL.md). The PR needs the `skip changeset` label applied by a maintainer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
There was a problem hiding this comment.
Pull request overview
This PR makes the Recommend integration tests GitHub Actions workflow resilient to forked pull requests, where GITHUB_TOKEN is read-only and write operations (labels/comments) otherwise fail with HTTP 403.
Changes:
- Add a
softWrite(operation, fn)helper to catch HTTP 403s from GitHub API write calls and log an informational message instead of failing the job. - Detect fork PRs and skip posting the “action required” comment for forks while keeping internal PR behavior (label + comment) intended to remain the same.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/recommend-integration-tests.yml | Soft-fails label/comment writes on 403 and suppresses comments for fork PRs to avoid misleading red checks. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 1
| const softWrite = async (operation, fn) => { | ||
| try { | ||
| await fn() | ||
| } catch (error) { | ||
| if (error.status === 403) { | ||
| core.info(`Skipped ${operation}: GITHUB_TOKEN cannot write to this repository from a forked PR. A maintainer can apply the '${INTEGRATION_LABEL_NAMES.recommended}' label manually if integration tests are warranted.`) | ||
| return | ||
| } | ||
| throw error |
|
Hi there @ianwinsemius! 👋 Thanks for taking the time to make this PR 🙏 I opened up an alternative idea for this over at: #7862, would this work for you, as well? This should skip the job entirely at the workflow level so you shouldn't run into the case where a Pull Request from a fork has a red status check for this workflow. Let me know what you think! |
|
Merged in: #7862 if that works for you @ianwinsemius! Closing this out but feel free to reach out if this isn't addressing the underlying need here 👀 |
Closes #
Problem
The
Recommend integration testsworkflow runs on everypull_requestevent, including PRs opened from forks. When triggered by a fork PR the providedGITHUB_TOKENhas read-only access to the upstream repo and any write call (addLabels/createComment) fails with:That failure surfaces as a red ❌
recommendcheck on every fork PR — even when the underlying source code is fine and all other checks (lint, tests, type-check, build, VRT, AAT, CodeQL) pass. It's misleading because the job's role is to suggest integration testing, not to gate the PR on its own infra success.Example: see the three PRs I have open (#7855, #7856, #7857) — all 34 functional checks pass, and only the
recommendworkflow fails because of this permission limitation.Fix
Wrap the label and comment API calls in a small helper that catches HTTP 403s and logs an actionable
core.infomessage instead of throwing. The job runs to completion green, and maintainers can apply theintegration-tests: recommendedlabel manually if the change warrants integration testing.Additionally, comment writes are skipped entirely on fork PRs (
context.payload.pull_request.head.repo.full_name !== context.payload.repository.full_name). The bot's comment recommends running an internal-only workflow that a fork PR author cannot trigger anyway — better to suppress the noise.Before / After
Before:
recommendjob fails withResource not accessible by integration→ red ❌ on the PR.After:
recommendjob catches the 403, logsSkipped addLabels: GITHUB_TOKEN cannot write to this repository from a forked PR. A maintainer can apply the 'integration-tests: recommended' label manually if integration tests are warranted.→ green ✅ on the PR.Changelog
New
Changed
.github/workflows/recommend-integration-tests.yml— wrap label/comment writes intry/catchkeyed on HTTP 403 so the job no longer fails on fork PRs.Removed
Rollout strategy
Testing & Reviewing
python3 -c "import yaml; yaml.safe_load(...)").softWrite(operation, fn)helper that distinguishes 403 from other errors.Merge checklist