BCR direct/transitive classification for BZLMOD mode on Bazel 7.1+ by bd-samratmuk · Pull Request #1812 · blackducksoftware/detect

bd-samratmuk · 2026-07-14T12:09:43Z

What

Adds a dedicated BCR extraction path for Bazel projects using BZLMOD mode on Bazel 7.1+. The new implementation leverages bazel mod graph --output json to build an accurate direct/transitive dependency graph for BCR-managed modules instead of flattening all modules as direct dependencies in the BOM.

Why

Previously, all BCR modules were added as direct root dependencies regardless of their actual relationship, resulting in an inaccurate dependency graph. This change preserves the true dependency structure exposed by Bazel's module graph.

How

Introduces BzlmodBcrExtractor to construct a direct/transitive dependency graph from bazel mod graph.
Adds BzlmodRepoMappingResolver to handle canonical repository names, aliases, and Bazel version-specific suffix formats.
Filters the graph to modules actually used by the requested target.
Resolves BCR modules via batched bazel mod show_repo calls.
Prevents duplicate flat-root additions from standard pipelines from overwriting dependency classification.
Adds comprehensive unit and end-to-end test coverage.

Backward Compatibility

No behavioural changes for WORKSPACE mode or BZLMOD projects on Bazel versions earlier than 7.1. The new path is enabled only when mode == BZLMOD and bazelVersion >= 7.1.

…SHOT

Copilot

Pull request overview

Adds a Bazel 7.1+ / BZLMOD-only extraction path that uses bazel mod graph --output json to preserve direct vs transitive relationships for BCR modules, instead of reporting BCR modules as a flat set of direct dependencies.

Changes:

Introduces BzlmodBcrExtractor + ModuleGraph parsing to build a parent/child dependency graph for BCR modules from bazel mod graph.
Adds BzlmodRepoMappingResolver to resolve canonical vs apparent repo labels and handle Bazel’s suffix variations (+ vs ~).
Expands documentation and adds unit + battery test fixtures for the new path.

Reviewed changes

Copilot reviewed 15 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/test/resources/battery/bazel/v2-bzlmod-bcr-extraction/MODULE.bazel	Test MODULE.bazel fixture declaring a BCR dep.
src/test/resources/battery/bazel/v2-bzlmod-bcr-extraction/bcr-target-scope-query.xout	Mock `bazel query` output used for target scoping.
src/test/resources/battery/bazel/v2-bzlmod-bcr-extraction/bcr-show-repo-batch.xout	Mock batched `bazel mod show_repo` output for URL resolution.
src/test/resources/battery/bazel/v2-bzlmod-bcr-extraction/bcr-mod-graph.xout	Mock `bazel mod graph --output json` output for tree structure.
src/test/resources/battery/bazel/v2-bzlmod-bcr-extraction/bcr-dump-repo-mapping.xout	Mock `bazel mod dump_repo_mapping ""` output for alias/suffix resolution.
src/test/resources/battery/bazel/v2-bzlmod-bcr-extraction/bazel-version-71.xout	Mock Bazel version output to gate the new path.
src/test/java/com/blackduck/integration/detect/battery/detector/BazelBattery.java	Adds a battery test covering the new BCR extraction path.
documentation/src/main/markdown/packagemgrs/bazel.md	Documents when/how direct/transitive classification is possible for Bazel deps.
detectable/src/test/java/com/blackduck/integration/detectable/detectables/bazel/v2/unit/BzlmodGraphJsonParserTest.java	Adds unit tests for tree-structure parsing into `ModuleGraph`.
detectable/src/test/java/com/blackduck/integration/detectable/detectables/bazel/v2/BzlmodRepoMappingResolverTest.java	Adds unit tests for label → module name resolution and suffix handling.
detectable/src/main/java/com/blackduck/integration/detectable/detectables/bazel/v2/ModuleGraph.java	New model object carrying direct keys + parent/child edges.
detectable/src/main/java/com/blackduck/integration/detectable/detectables/bazel/v2/BzlmodRepoMappingResolver.java	New resolver for canonical/apparent label mapping and suffix detection.
detectable/src/main/java/com/blackduck/integration/detectable/detectables/bazel/v2/BzlmodGraphJsonParser.java	Adds `parseModuleGraph` to preserve parent/child structure from mod graph JSON.
detectable/src/main/java/com/blackduck/integration/detectable/detectables/bazel/v2/BzlmodBcrExtractor.java	New BCR extractor building a structured dependency graph using mod graph + show_repo.
detectable/src/main/java/com/blackduck/integration/detectable/detectables/bazel/v2/BazelV2Extractor.java	Wires in the BZLMOD Bazel 7.1+ BCR path and dedupes pipeline results by ExternalId.
build.gradle	Version bump.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

bd-samratmuk · 2026-07-14T17:59:33Z

+    public String stripCanonicalSuffix(String rawName) {
+        return stripSuffix(rawName, detectedSuffix);
+    }


addressed in the latest commit.

bd-samratmuk · 2026-07-14T17:59:38Z

+            String repoArg = moduleKeyToRepoArg.get(moduleKey);
+            List<String> showRepoArgs = BazelQueryBuilder.mod().showRepoRaw(repoArg).build();
+            Optional<String> output = bazelCmd.executeModCommandToString(showRepoArgs);
+            if (output.isPresent() && !output.get().trim().isEmpty()) {
+                result.put(moduleKey, output.get());
+            } else {
+                logger.debug("BZLMOD BCR: per-module show_repo produced no output for '{}'", moduleKey);
+            }


addressed in the latest commit.

karolynbd · 2026-07-14T15:16:37Z

+Bazel external dependencies fall into three categories that differ in how much structural information Bazel records about them.
+
+**http_archive / git_repository / go_repository (WORKSPACE rules)**
+Raw source-archive fetches. There is no parent-child relationship — every dependency (direct and transitive alike) must be manually declared in the WORKSPACE file, so it is always a flat list. [detect_product_short] discovers these deps but cannot classify them as direct or transitive.


I would use "dependencies" instead of "deps"

addressed in the latest commit.

karolynbd · 2026-07-14T15:24:32Z

+| maven_install (rules_jvm_external) | No — managed internally by rules_jvm_external | All deps reported as direct |
+| BCR modules (bazel_dep, BZLMOD 7.1+) | Yes — `bazel mod graph` exposes full tree | Direct and transitive correctly classified |
+
+For WORKSPACE http_archive deps, the transitive closure was never recorded — the developer had to write every dependency explicitly into the WORKSPACE, so there is nothing to reconstruct. For maven_install, the resolution tree exists inside rules_jvm_external's lockfile but is not surfaced through Bazel's module graph API. BCR modules are different: `bazel mod graph --output json` returns the exact parent-child edges, which [detect_product_short] uses to produce a classified BOM.


These last 2 paragraphs seem duplicate and/or can be combined with the 1st 3 paragraphs?

addressed in the latest commit.

karolynbd · 2026-07-15T13:10:12Z


+### Processing for BCR modules (BZLMOD mode, Bazel 7.1+)
+
+When [detect_product_short] detects BZLMOD mode on Bazel 7.1 or later, it runs a dedicated BCR extraction path before the standard pipelines. The http_archive pipeline still runs afterward, but any dep already classified by the BCR extractor is deduplicated by ExternalId — this preserves the direct/transitive edges the BCR extractor built and avoids promoting transitive BCR deps to direct. Custom http_archive rules absent from `MODULE.bazel` (private repos, non-BCR archives) are unaffected and still added to the BOM flat.


Not sure you realize this will say: "When Detect detects". Sounds better with a different verb.

nice catch! addressed in the latest commit. @karolynbd

shantyk

Thank you for all the javadocs -- they were super useful during the review.

cpottsbd · 2026-07-22T13:53:16Z


+## BCR Modules and Dependency Classification
+
+Bazel external dependencies fall into three categories that differ in how much structural information Bazel records about them.


Suggestion:
Bazel supports several mechanisms for consuming external dependencies, which differ in the amount of dependency metadata and relationship information available to [detect_product_short] and affects how [detect_product_short] can classify dependencies as direct or transitive.

cpottsbd · 2026-07-22T14:00:55Z

+
+Bazel external dependencies fall into three categories that differ in how much structural information Bazel records about them.
+
+**http_archive / git_repository / go_repository (WORKSPACE rules)**


Suggestion for this section (Feel free to edit):

WORKSPACE-Based http_archive, git_repository, and go_repository
External source code is fetched directly into the build environment. These rules provide no dependency relationship metadata beyond the declarations present in the WORKSPACE file.

All dependencies must be explicitly declared by the project as Bazel does not maintain a dependency graph for these repository types. The resulting dependency list is inherently flat, with no parent-child relationships available for analysis or reconstruction. As a result, [detect_product_short] can identify these dependencies but reports all of them as direct dependencies.

cpottsbd · 2026-07-22T14:03:58Z

+**http_archive / git_repository / go_repository (WORKSPACE rules)**
+Raw source-archive fetches. There is no parent-child relationship — every dependency must be manually declared in the WORKSPACE file, so the list is always flat with no transitive structure to reconstruct. [detect_product_short] discovers these dependencies but reports them all as direct.
+
+**maven_install (rules_jvm_external)**


Suggestion for this section, edit as needed:

Maven Dependencies via rules_jvm_external
Projects that use maven_install from rules_jvm_external rely on Maven's dependency resolution model. The complete dependency graph exists internally within the resolver and its lockfile infrastructure, however that information is not exposed through Bazel's module graph APIs.

[detect_product_short] can extract Maven coordinates from the Bazel build configuration, but it cannot accurately determine dependency ancestry. Dependencies obtained through maven_install are therefore reported as a flat list rather than a hierarchical dependency tree.

cpottsbd · 2026-07-22T14:06:43Z

+**BCR modules (bazel_dep in MODULE.bazel)**
+The only category where Bazel owns and records the full dependency tree. Each module published to the [Bazel Central Registry (BCR)](https://registry.bazel.build/) carries its own `MODULE.bazel` that declares only its own direct dependencies — the same model npm, Maven, and Cargo use. Bazel resolves the full transitive graph by reading each module's `MODULE.bazel` recursively and exposes the result via `bazel mod graph --output json` (Bazel 7.1+), which includes explicit parent-child edges. [detect_product_short] uses these edges to produce a classified BOM where direct and transitive dependencies are correctly distinguished.
+
+For BZLMOD projects on Bazel 7.1+, [detect_product_short] calls `bazel mod graph` first to capture the tree structure, then `bazel mod show_repo` per module to resolve source URLs. maven_install and http_archive results are added to the same BOM but remain flat, since no tree is available for those types.


Tweak suggestion:
For Bzlmod-based projects running Bazel 7.1 or later, [detect_product_short] first executes bazel mod graph to obtain the complete module dependency hierarchy. [detect_product_short] then invokes bazel mod show_repo for each module to resolve repository source locations and associated metadata.

Dependencies discovered are incorporated into the same BOM, however, because these dependency types do not expose equivalent graph information through Bazel, they are reported as flat dependency sets.

cpottsbd · 2026-07-22T14:08:50Z

+**maven_install (rules_jvm_external)**
+Maven coordinates managed through rules_jvm_external. The resolution tree exists inside rules_jvm_external's own lockfile machinery but is not surfaced through Bazel's module graph API. [detect_product_short] reads coordinates from the Bazel build graph and reports them flat.
+
+**BCR modules (bazel_dep in MODULE.bazel)**


Suggestion:
Bazel Central Registry (BCR) Modules (bazel_dep in MODULE.bazel)

cpottsbd · 2026-07-22T14:15:08Z

+Maven coordinates managed through rules_jvm_external. The resolution tree exists inside rules_jvm_external's own lockfile machinery but is not surfaced through Bazel's module graph API. [detect_product_short] reads coordinates from the Bazel build graph and reports them flat.
+
+**BCR modules (bazel_dep in MODULE.bazel)**
+The only category where Bazel owns and records the full dependency tree. Each module published to the [Bazel Central Registry (BCR)](https://registry.bazel.build/) carries its own `MODULE.bazel` that declares only its own direct dependencies — the same model npm, Maven, and Cargo use. Bazel resolves the full transitive graph by reading each module's `MODULE.bazel` recursively and exposes the result via `bazel mod graph --output json` (Bazel 7.1+), which includes explicit parent-child edges. [detect_product_short] uses these edges to produce a classified BOM where direct and transitive dependencies are correctly distinguished.


Suggestion:

Dependencies declared with bazel_dep in MODULE.bazel provide the most complete dependency metadata available within Bazel. Modules published to the Bazel Central Registry (BCR) include their own MODULE.bazel that declare their own direct dependencies, following the same dependency management model used by Maven, npm, and Cargo. Beginning with Bazel 7.1, Bazel resolves the full transitive graph by reading each module's MODULE.bazel recursively and exposes the result via bazel mod graph --output json, which includes explicit parent-child relationships between modules. [detect_product_short] consumes this data to accurately classify dependencies as either direct or transitive when generating a Bill of Materials (BOM).

cpottsbd · 2026-07-22T14:16:04Z

+[detect_product_short] parses the `show_repo` output for GitHub URLs using the same extraction logic as the standard http_archive BZLMOD pipeline.
+
+**Step 3 — Build the classified BOM:**
+Modules declared via `bazel_dep` appear as direct dependencies. Their transitive deps appear nested under their respective parents rather than at the root. The BOM correctly reflects which components your project explicitly chose versus which came along transitively.


Minor tweak:
Modules declared via bazel_dep appear as direct dependencies. Their transitive dependencies appear nested under their respective parents rather than at the root. The BOM correctly reflects which components your project explicitly chose versus which came along transitively.

cpottsbd · 2026-07-22T14:18:07Z

+**Step 3 — Build the classified BOM:**
+Modules declared via `bazel_dep` appear as direct dependencies. Their transitive deps appear nested under their respective parents rather than at the root. The BOM correctly reflects which components your project explicitly chose versus which came along transitively.
+
+Modules for which no GitHub URL can be extracted (private repos, custom non-BCR rules) are logged at WARN and excluded from the BOM. The warning includes the raw URL(s) found so you can investigate what was missed.


Tweak:
A WARN-level log entry is written for Modules where no GitHub URL can be extracted (private repos, custom non-BCR rules), and excluded from the BOM. To assist in investigating what was missed, the warning includes the raw URL(s) found.

cpottsbd · 2026-07-22T14:23:04Z


+### Processing for BCR modules (BZLMOD mode, Bazel 7.1+)
+
+When [detect_product_short] scans with BZLMOD mode on Bazel 7.1 or later, it runs a dedicated BCR extraction path before the standard pipelines. The http_archive pipeline still runs afterward, but any dep already classified by the BCR extractor is deduplicated by ExternalId — this preserves the direct/transitive edges the BCR extractor built and avoids promoting transitive BCR deps to direct. Custom http_archive rules absent from `MODULE.bazel` (private repos, non-BCR archives) are unaffected and still added to the BOM flat.


Suggestion:

When [detect_product_short] runs in Bzlmod mode on Bazel 7.1 or later, it performs a dedicated Bazel Central Registry (BCR) dependency extraction step before executing the standard dependency discovery pipelines.

After BCR extraction completes, the http_archive discovery pipeline still runs to identify additional external dependencies. However, any dependency that was already identified and classified through the BCR module graph is deduplicated using its ExternalId. This ensures that the dependency relationships established by the BCR extractor are preserved, including the distinction between direct and transitive dependencies.

Without this deduplication step, dependencies discovered later through http_archive analysis could be incorrectly reclassified as direct dependencies, resulting in the loss of the dependency hierarchy derived from the Bzlmod module graph.

Custom http_archive definitions, private repositories, and non-BCR source archives absent from MODULE.bazel are not affected by this process. These dependencies continue to be discovered through the standard repository-rule analysis and are added to the BOM as flat dependency entries.

cpottsbd · 2026-07-22T14:23:38Z


 This dual approach ensures that the Bazel tool works seamlessly for both modern bzlmod-based and traditional WORKSPACE-based Bazel projects, always probing the graph to decide which pipeline to run.

+### Processing for BCR modules (BZLMOD mode, Bazel 7.1+)


Tweak suggestion:

Processing for BCR modules (Bzlmod mode on Bazel 7.1 or later)

bd-samratmuk · 2026-07-22T16:37:36Z

 Without this deduplication step, dependencies discovered later through http_archive analysis could be incorrectly reclassified as direct dependencies, resulting in the loss of the dependency hierarchy derived from the Bzlmod module graph.

-Custom http_archive definitions, private repositories, and non-BCR source archives absent from `MODULE.bazel` are not affected by this process. These dependencies continue to be discovered through the standard repository-rule analysis and are added to the BOM as flat dependency entries.
+Custom http_archive definitions, private repositories, and non-BCR source archives absent from `MODULE.bazel` are not part of the BCR classification step. These dependencies are processed by the standard repository-rule analysis pipeline. However, that pipeline can only extract dependencies whose URLs match known GitHub release patterns. Private repositories, custom repository macros, and non-standard registry URLs will produce a WARN-level log entry and will be excluded from the BOM. See [Using Bazel Fetch for Additional Coverage](#using-bazel-fetch-for-additional-coverage) for a workaround in these environments.


@cpottsbd had to tweak your suggestion a bit to maintain parity

dterrybd · 2026-07-22T17:30:14Z

I believe there are some battery tests that cover some of this but I'd be more comfortable if we could add a few unit tests around this major new piece of functionality.

Addressed in the latest commit.

dterrybd · 2026-07-22T17:42:50Z

+        test.property("detect.bazel.target", "//src/main:example");
+        test.property("detect.bazel.mode", "BZLMOD");
+        test.property("detect.bazel.dependency.sources", "HTTP_ARCHIVE");
+        test.executableFromResourceFiles(


The comments differ here but it looks like 5 & 7 are the same and 4 & 6 are the same? Or is there something else going on here?

Nice catch! Let me explain this. The BCR extraction step (steps 4 & 5) and the HTTP_ARCHIVE pipeline (steps 6 & 7) genuinely run the same two Bazel commands against the same target. Steps 4 & 6 both invoke bazel query kind(.*library, deps(//src/main:example)) and steps 5 & 7 both invoke bazel mod show_repo @@protobuf~ @@abseil-cpp~. Running the same Bazel command twice has no real cost since Bazel caches the results. The key thing being tested is the ExternalId deduplication filter. By the time HTTP_ARCHIVE runs, BzlmodBcrExtractor.getResolvedExternalIds() already contains protobuf and abseil-cpp. The HTTP pipeline checks this set and suppresses both results, so nothing is re-added to the BOM. Without this filter, the HTTP pipeline would re-add those deps as flat root children and overwrite the direct/transitive hierarchy the BCR step established. The duplicate fixture files are the assertion for the complete end-to-end flow. If the dedup filter ever stopped working, the HTTP pipeline would re-add both deps as direct deps, the BDIO output would differ, and the battery test would fail. We have also added BzlmodBcrExtractorTest at the unit level which directly verifies that getResolvedExternalIds() is correctly populated, making the intent more explicit.

bd-samratmuk and others added 6 commits July 9, 2026 14:32

add bzlmod transitive classification

8328f1b

use dump_repo_mapping

b41d6ac

Release 12.0.0-SIGQA11-IDETECT-5149

9f26b79

Using the next snapshot post release 12.0.0-SIGQA12-IDETECT-5149-SNAP…

615e54d

…SHOT

add battery test

7e92149

add unit tests

07cb5bc

bd-samratmuk requested review from Copilot and cpottsbd July 14, 2026 12:09

Copilot started reviewing on behalf of bd-samratmuk July 14, 2026 12:10 View session

bd-samratmuk requested a review from karolynbd July 14, 2026 12:10

Copilot AI reviewed Jul 14, 2026

View reviewed changes

karolynbd reviewed Jul 14, 2026

View reviewed changes

bd-samratmuk requested a review from bd-vbm July 14, 2026 15:37

address copilot review comments

31a2695

karolynbd reviewed Jul 15, 2026

View reviewed changes

shantyk approved these changes Jul 22, 2026

View reviewed changes

edit doc

0ce88dc

cpottsbd requested changes Jul 22, 2026

View reviewed changes

bd-samratmuk added 3 commits July 22, 2026 21:54

address review comments for doc

7e23ce5

update release notes

80736ce

fix doc

ede0208

bd-samratmuk commented Jul 22, 2026

View reviewed changes

bd-samratmuk requested a review from dterrybd July 22, 2026 16:40

dterrybd approved these changes Jul 22, 2026

View reviewed changes

cpottsbd approved these changes Jul 22, 2026

View reviewed changes

bd-samratmuk added 2 commits July 23, 2026 14:27

add unit test & organize command responses

7b86f53

Merge branch 'master' into IDETECT-5149

a122521

bd-samratmuk merged commit 6491b61 into master Jul 23, 2026


		### Processing for BCR modules (BZLMOD mode, Bazel 7.1+)

		When [detect_product_short] detects BZLMOD mode on Bazel 7.1 or later, it runs a dedicated BCR extraction path before the standard pipelines. The http_archive pipeline still runs afterward, but any dep already classified by the BCR extractor is deduplicated by ExternalId — this preserves the direct/transitive edges the BCR extractor built and avoids promoting transitive BCR deps to direct. Custom http_archive rules absent from `MODULE.bazel` (private repos, non-BCR archives) are unaffected and still added to the BOM flat.


		## BCR Modules and Dependency Classification

		Bazel external dependencies fall into three categories that differ in how much structural information Bazel records about them.


		Bazel external dependencies fall into three categories that differ in how much structural information Bazel records about them.

		http_archive / git_repository / go_repository (WORKSPACE rules)


		### Processing for BCR modules (BZLMOD mode, Bazel 7.1+)

		When [detect_product_short] scans with BZLMOD mode on Bazel 7.1 or later, it runs a dedicated BCR extraction path before the standard pipelines. The http_archive pipeline still runs afterward, but any dep already classified by the BCR extractor is deduplicated by ExternalId — this preserves the direct/transitive edges the BCR extractor built and avoids promoting transitive BCR deps to direct. Custom http_archive rules absent from `MODULE.bazel` (private repos, non-BCR archives) are unaffected and still added to the BOM flat.


		This dual approach ensures that the Bazel tool works seamlessly for both modern bzlmod-based and traditional WORKSPACE-based Bazel projects, always probing the graph to decide which pipeline to run.

		### Processing for BCR modules (BZLMOD mode, Bazel 7.1+)

Uh oh!

Conversation

bd-samratmuk commented Jul 14, 2026

What

Why

How

Backward Compatibility

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shantyk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Processing for BCR modules (Bzlmod mode on Bazel 7.1 or later)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants