ci: enable best-effort delta builds using ADO pipelines#17684
Open
dmcilvaney wants to merge 1 commit into
Open
ci: enable best-effort delta builds using ADO pipelines#17684dmcilvaney wants to merge 1 commit into
dmcilvaney wants to merge 1 commit into
Conversation
Contributor
Author
|
Note: Planning on renaming the file to something more accurate, but will need to combine that with ADO changes, leaving as-is to allow testing in the meantime. |
PawelWMS
reviewed
Jun 10, 2026
dmcilvaney
force-pushed
the
damcilva/4.0/pipelines/enable_v1_delta_build
branch
from
b1f5548 to
ed24ecf
Compare
Automatic source upload is being reworked, but we don't need to wait on that to enable the initial version of delta builds. Disable the source upload component in the delta build pipeline, and configure it to calculate the commit range using the ADO REST API instead of git commands (since this will be post-merge). Also fix a bug where the environment variable names were shadowed by ADO predefined variables.
dmcilvaney
force-pushed
the
damcilva/4.0/pipelines/enable_v1_delta_build
branch
from
ed24ecf to
11afe71
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Enables best-effort post-merge “delta” package builds in Azure DevOps by introducing a shared step template and a new helper that determines (target, source) commit ranges via the Azure DevOps control-plane API (rather than git heuristics), while also removing the package-build submission from the legacy source-upload pipeline and avoiding env-var name shadowing.
Changes:
- Add
determine_commit_range.py(Azure DevOps SDK-based) plus docs/deps to compute a stable, non-overlapping commit range for delta builds. - Introduce a shared ADO step sub-template (
common-steps.yml) and refactor stages templates to consume it. - Add a new
package-buildADO wrapper + stages template for post-merge delta builds; adjust lock verification expectations accordingly.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/ci/control-tower/verify_locks.sh |
Stops self-unshallowing; relies on the pipeline’s shared “Ensure full git history” step. |
scripts/ci/ado/requirements.txt |
Pins Azure DevOps Python SDK dependency for ADO control-plane queries. |
scripts/ci/ado/README.md |
Documents conventions and caller contract for ADO helper scripts. |
scripts/ci/ado/determine_commit_range.py |
New helper to compute commit range using ADO build history (best-effort fallback included). |
ruff.toml |
Adds per-file ignore to allow namespace-style execution of scripts/ci/** helpers without __init__.py. |
.github/workflows/ado/templates/steps/common-steps.yml |
New shared step template: unshallow, install deps, determine range, verify locks, compute change set, verify rendered specs. |
.github/workflows/ado/templates/sources-upload-stages.yml |
Refactors to use common-steps.yml, removes package-build submission, and fixes reserved env var shadowing. |
.github/workflows/ado/templates/package-build-stages.yml |
New raw stages template for post-merge delta package builds via Control Tower. |
.github/workflows/ado/sources-upload.yml |
Updates wrapper metadata and parameters (e.g., artifact base name). |
.github/workflows/ado/package-build.yml |
New wrapper pipeline wiring for the delta package-build flow. |
.github/instructions/ado-pipeline.instructions.md |
Extends applyTo glob and documents shared step sub-templates + ADO control-plane auth via System.AccessToken. |
Comment on lines
+10
to
+15
| # Keeping these as STEPS (not a separate job/stage) is deliberate: steps 3-5 | ||
| # emit job-scoped pipeline variables (sourceCommit, targetCommit, | ||
| # changedComponentsFile, renderSetFile) and write the change-set files to disk. | ||
| # Those flow freely to later steps in the SAME job but would need output | ||
| # variables + artifact upload/download to cross a job boundary. Splicing the | ||
| # common steps inline keeps that wiring trivial. |
Comment on lines
+17
to
+25
| # Contract for the including job: | ||
| # * It MUST define the job-scope variable `ob_outputDirectory` (both stages | ||
| # templates set it from their `outputDirectory` parameter); the verify-locks | ||
| # and prepare-change-set steps publish triage artifacts there. | ||
| # * It MUST be fail-loud (no job-level `continueOnError`). These steps rely on | ||
| # a failed step stopping the job, so later steps assume earlier ones | ||
| # succeeded -- e.g. the render step trusts that "Prepare change set" ran and | ||
| # set renderSetFile to a real path. | ||
| # |
| # ADO task: 19179 | ||
| continueOnError: true | ||
| # Must exceed the script's --poll-timeout-seconds (default 7200s = 120m) | ||
| # Must exceed the script's --poll-timeout-seconds (default 600s = 10m) |
Comment on lines
+75
to
+81
| --api-base-url "$API_BASE_URL" \ | ||
| --build-reason "$CT_BUILD_REASON" \ | ||
| --changed-components-file "$CHANGED_COMPONENTS_FILE" \ | ||
| --package-target ${{ parameters.packageTarget }} \ | ||
| --official-build \ | ||
| --commit-sha "$SOURCE_COMMIT" \ | ||
| --repo-uri "$UPSTREAM_REPO_URL" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automatic source upload is being reworked, but we don't need to wait on that to enable the initial version of delta builds. Disable the source upload component in the delta build pipeline, and configure it to calculate the commit range using the ADO REST API instead of git commands (since this will be post-merge).
Also fix a bug where the environment variable names were shadowed by ADO predefined variables.