Use per-language CodeQL bundles by henrymercer · Pull Request #4146 · github/codeql-action

henrymercer · 2026-09-15T17:18:12Z

Download self-contained per-language bundles for eligible analyses on GitHub.com. Selection requires one explicitly configured language, a GitHub-hosted runner, zstd, and a supported language/platform combination. Release bundles require CodeQL 2.27.1 or newer; nightlies use the same eligibility rules without the release-version check. The per_language_bundles feature flag remains off by default.

Extract per-language bundles into the runner's temporary directory rather than the toolcache. If an automatically selected asset returns 404, fall back to the combined bundle from the same release. Explicit per-language tools URLs are not substituted.

Add bundle-language and fallback telemetry, plus generated nightly PR checks for all ten languages covering extractor contents, toolcache isolation, database creation and analysis. Update existing cache-dependent checks to request multiple languages so they continue to use the combined bundle.

Risk assessment

Low risk: Automatic per-language bundle selection is feature-flagged and off by default. The unflagged behavior is limited to rare, explicitly supplied per-language bundle URLs and prevents those partial installs from entering the toolcache.

Which use cases does this change impact?

Workflow types:

Advanced setup
Managed

Products:

Code Scanning
Code Quality
Other first-party - Analyses using the shared CodeQL setup path.

Environments:

Dotcom - Automatic per-language selection on GitHub.com; explicit bundle URL handling also applies to data-residency environments.
GHES - Explicit per-language bundle URL handling only. Automatic selection remains disabled.

How did/will you validate this change?

Unit tests - Eligibility, bundle selection, version reporting, fallback, and toolcache isolation.
End-to-end tests - Generated nightly PR checks cover all ten languages. These still need to run for this PR.

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

Feature flags - Disable automatic selection with per_language_bundles.
Rollback - Release a fix or revert if explicit bundle URL handling regresses; that path is not controlled by the flag.

How will you know if something goes wrong after this change is released?

Telemetry - Monitor per-language downloads, observed combined-bundle fallbacks, download timings and initialization failures through the rollout dashboard.

Are there any special considerations for merging or releasing this change?

Special considerations - Merge the bundle-resolution prerequisite, #4145, first. Keep the feature flag disabled until the end-to-end checks pass, matching assets are available, and telemetry consumer support is deployed.

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Consider adding a changelog entry for this change.
Confirm the readme and docs have been updated if necessary.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Warning

Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Copilot review overview

🟡 Changes recommended

Fallback telemetry underreports total setup duration by excluding the failed per-language request.

Get a fresh assessment by requesting another Copilot review.

Review tier: Balanced
Findings: 1

Open (1)

Include the failed attempt in total download duration · New

What changed in this PR

Adds feature-gated per-language CodeQL bundle selection, extraction, fallback, telemetry, and validation.

Changes:

Selects eligible release/nightly per-language bundles with 404 fallback.
Keeps partial bundles out of the toolcache and reports telemetry.
Adds unit and generated end-to-end validation across ten languages.

File	Description
`src/per-language-bundles.ts`	Implements bundle eligibility and URL recognition.
`src/per-language-bundles.test.ts`	Tests eligibility and recognition.
`src/setup-codeql.ts`	Resolves, downloads, caches, and falls back between bundles.
`src/setup-codeql.test.ts`	Tests selection, fallback, versions, and caching.
`src/tools-download.ts`	Extends download telemetry.
`src/status-report.ts`	Defines telemetry fields.
`src/init-action.ts`	Reports init telemetry.
`src/setup-codeql-action.ts`	Reports setup telemetry.
`src/feature-flags.ts`	Adds the disabled-by-default feature flag.
`pr-checks/sync.ts`	Supports custom generated matrices.
`pr-checks/checks/per-language-bundle-validation.yml`	Validates all supported language bundles.
`pr-checks/checks/bundle-toolcache.yml`	Preserves combined-bundle cache testing.
`.github/workflows/codescanning-config-cli.yml`	Ensures reusable combined-bundle caching.
`lib/entry-points.js`	Generated artifact; content excluded.
`.github/workflows/__bundle-toolcache.yml`	Generated workflow; content excluded.
`.github/workflows/__per-language-bundle-validation.yml`	Generated workflow; content excluded.

Files excluded by content exclusion policy (3)

.github/workflows/__bundle-toolcache.yml
.github/workflows/__per-language-bundle-validation.yml
lib/entry-points.js

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

mbg

I have had a look through the changes here and added quite a few detailed comments. Additionally, some high-level ones:

Like with the other PR this is stacked on, this was a lot more difficult (and time-consuming) to review than it needed to be, because all of the changes were contained in one commit (except the follow-up to Copilot's review comment). There were clear opportunities for breaking the changes up and structuring them to be more incremental.
Quite a few of the function-level comments and test names make sense in the context of this PR where it's obvious that they relate to per-language vs combined bundles, but wouldn't make sense outside of this context. Some of my review comments highlight specific cases of this, but I didn't comment on all of them.
Try and remember to make use of the new ActionState infrastructure whenever possible in new or changed code. In particular, the getPerLanguageBundleLanguage function would benefit from this since the tests would not need to be serial.

mbg · 2026-09-16T08:48:50Z

+  if (!isGitHubHostedRunner()) {
+    // Per-language installs stay out of the toolcache; self-hosted runners should retain
+    // the reusable combined bundle instead.
+    return explain("the job is not running on a GitHub-hosted runner");
+  }


I am wondering if this is the correct choice. Self-hosted runners could use custom images / persistent storage to pre-install the desired CLI bundle in the toolcache. If that's not available, wouldn't it be more desirable to download the language-specific bundle for improved performance like on GH-hosted runners?

Self-hosted runner can indeed preinstall CodeQL, but I don't think we should assume it. I would prefer to have a better way to determine whether the toolcache is persistent or not before rolling out to self-hosted runners to avoid a situation where we install many per-language bundles and use an unnecessary amount of disk space.

mbg · 2026-09-16T09:44:24Z

+  const client = github.getOctokit("123", {
+    request: {
+      fetch: async () =>
+        new Response(JSON.stringify([{ tag_name: tagName }]), {
+          headers: { "content-type": "application/json" },
+        }),
+    },
+  });


This is less precise than the stubbing that stubHostedNightly replaces. Consider whether to stub sinon.stub(client.rest.repos, "listReleases"); instead of request.

As far as I can tell, this hasn't been addressed, but that's not blocking.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

henrymercer · 2026-09-16T19:24:20Z

Lots of good ideas, thank you for the detailed review!

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

mbg

Thanks for tackling all the review comments I left during the previous round! I have reviewed the new commits up to and including 2f552a9. I'll look at the remaining ones later this evening, but figured it would be good to post what I have so far now in the interest of time.

Most of the comments I added so far are minor, but there are a couple where I think the changes haven't quite been what I intended with my comments.

As a high-level point, I appreciate that the feedback is addressed across multiple commits, but the breakdown is still not ideal. Most of the new commits I have reviewed so far have a description that suggests they address one point of feedback, but actually mix in other smaller changes as well (some of which address other feedback and some of which are unrelated to the previous review). That makes it harder to verify whether a particular piece of feedback was addressed because more context switching is required while reviewing the individual commits and it takes a bit more effort to identify whether a given change relates to the main point of feedback or is unrelated.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

mbg

I have now reviewed the remaining commits from before my last review, as well as the ones in which you addressed my feedback from yesterday. Thanks for taking care of all that!

There are a few very minor points that are left, none of which are blocking / all of which can be followed-up on later. I am happy to approve this PR in this state and I'll go ahead and merge it.

mbg · 2026-09-18T11:59:11Z

  return databaseBundlePath;
 }

+/** Returns the elapsed milliseconds, rounded, since a `performance.now()` timestamp. */


Minor: "since a performance.now() timestamp" is incomplete

mbg · 2026-09-18T12:01:11Z

  const language = await getPerLanguageBundleLanguage(
-    {
+    initAllState({
      env: getTestEnv(),


Minor: not required if it's not customised. initAllState will automatically call getTestEnv() to initialise env.

mbg · 2026-09-18T12:02:08Z

+    initAllState({
      env,
      features: createFeatures(enabledFeatures),
      logger: getRecordingLogger([], { logToConsole: false }),


Minor: Not needed if you don't inspect the log output. initAllState will automatically initialise a logger.

mbg · 2026-09-18T12:10:12Z


  // Check whether per-language bundles are published for the requested CLI version.
-  // Skip this for the latest nightly, whose tag contains a date rather than a CLI version.
+  // Latest-nightly selection skips this release-version check, but not the other eligibility checks.


Minor: The comment now again doesn't explain why it's OK to skip.

Suggested change

// Latest-nightly selection skips this release-version check, but not the other eligibility checks.

// If `nightly` tools were requested, skip the version check since we don't have one at this point.

mbg · 2026-09-18T12:19:55Z

+  const client = github.getOctokit("123", {
+    request: {
+      fetch: async () =>
+        new Response(JSON.stringify([{ tag_name: tagName }]), {
+          headers: { "content-type": "application/json" },
+        }),
+    },
+  });


As far as I can tell, this hasn't been addressed, but that's not blocking.

Use per-language CodeQL bundles

ec5eecb

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

henrymercer requested a balanced review from Copilot September 15, 2026 17:18

github-actions Bot added the size/XXL May be extremely hard to review label Sep 15, 2026

Copilot started reviewing on behalf of henrymercer September 15, 2026 17:19 View session

henrymercer added this pull request to stack #4147 September 15, 2026 17:19

Copilot AI reviewed Sep 15, 2026

View reviewed changes

Comment thread src/setup-codeql.ts Outdated

Include failed bundle attempts in fallback timing

eb76062

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

henrymercer marked this pull request as ready for review September 15, 2026 18:57

henrymercer requested a review from a team as a code owner September 15, 2026 18:57

mbg reviewed Sep 16, 2026

View reviewed changes

henrymercer and others added 10 commits September 16, 2026 16:56

Use per-language CodeQL bundles

289376d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Include failed bundle attempts in fallback timing

59ce3a2

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Isolate per-language bundle eligibility state

2d47caf

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Share CodeQL bundle platform definitions

dfb9bf5

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Centralize CodeQL download telemetry fields

f536ef4

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Clarify bundle resolution and latest-nightly selection

2f552a9

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Extract explicit CodeQL bundle URL classification

bd2ddba

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Return toolcache rejection reasons with Result

89606bb

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Share elapsed-time rounding for bundle downloads

f4fa111

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Use Result.orElse for bundle extraction paths

69f4715

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Merge published per-language bundle history

5a2f29e

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Base automatically changed from henrymercer/bundle-resolution-pr to main September 17, 2026 14:43

mbg reviewed Sep 17, 2026

View reviewed changes

henrymercer and others added 6 commits September 17, 2026 19:00

Use shared test state for bundle eligibility

28b8f59

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Group bundle eligibility test state overrides

ed3a24c

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Simplify per-language platform eligibility checks

549d498

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Rename the platform module

ead1f7d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Move download telemetry into the status-report directory

79fe3a1

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Share per-language telemetry fields without renaming

ecec9b5

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

henrymercer and others added 2 commits September 17, 2026 19:07

Describe the bundle URL resolver

f18f353

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Clarify the latest-nightly eligibility exception

07fa87d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

mbg approved these changes Sep 18, 2026

View reviewed changes

mbg added this pull request to the merge queue Sep 18, 2026

Merged via the queue into main with commit a65b83a Sep 18, 2026
233 checks passed

mbg deleted the henrymercer/per-language-bundles-pr branch September 18, 2026 12:43

github-actions Bot mentioned this pull request Sep 18, 2026

Merge main into releases/v4 #4152

Merged

8 tasks

mbg added a commit that referenced this pull request Sep 18, 2026

Add changelog entry for #4146

a97cdca

github-actions Bot mentioned this pull request Sep 18, 2026

Merge releases/v4 into releases/v3 #4154

Merged

6 tasks

	// Latest-nightly selection skips this release-version check, but not the other eligibility checks.
	// If `nightly` tools were requested, skip the version check since we don't have one at this point.

Conversation

henrymercer commented Sep 15, 2026

Risk assessment

Which use cases does this change impact?

How did/will you validate this change?

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

How will you know if something goes wrong after this change is released?

Are there any special considerations for merging or releasing this change?

Merge / deployment checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

mbg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

henrymercer commented Sep 16, 2026

Uh oh!

mbg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mbg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants