Skip to content

test(e2e): hermetic end-to-end coverage of a real git merge - #546

Open
behinddwalls wants to merge 1 commit into
sq/orchestrator-profilesfrom
sq/git-e2e
Open

test(e2e): hermetic end-to-end coverage of a real git merge#546
behinddwalls wants to merge 1 commit into
sq/orchestrator-profilesfrom
sq/git-e2e

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The existing E2E suite runs Runway on the noop merger, so a request reaching landed proves the pipeline's choreography and nothing at all about git. Every risky part of merging for real — the git binary in the image, checkout provisioning, the apply and push, head-branch updates — had no automated coverage through the stack.

None of that needs a provider. The merger speaks git:// against any remote, so pointing it at a bare repository on a shared volume exercises the whole merge path with no credential, no network, and no account anywhere — which is what lets these assertions gate a pull request.

What?

A second suite lands git:// changes and asserts against the repository itself rather than against request status: which commits reached the target, in what order, and in how many ref updates. A three-change stack must arrive in exactly one ref update, counted from the target's reflog — the property that distinguishes a submit queue from merging changes one at a time, and one that is hard to check against a live provider.

It also pins two behaviors worth not regressing: a conflicting change leaves the target untouched, and re-submitting a change after it has landed is rejected as stale (landing moved its head branch, so the URI no longer describes where that branch points).

testutil.NewComposeStack gained WithOverlay, so the suite states only what differs from the base stack — Runway's merge target — instead of copying the whole service definition. A copy is the thing that rots: services added to the base file never reach the duplicate, and nothing fails until the variant runs.

The suite drives the bare repository with the same pinned git the merger uses, rather than the host's, since these assertions depend on repository mechanics and an ambient git brings ambient configuration with it.

Test Plan

make e2e-git-test — the new suite.

make e2e-test — both suites; the existing one is unchanged and still passes.

Also fixes make help, whose target regex excluded digits, so neither e2e-test nor e2e-git-test was ever listed.

Stack

  1. feat(runway): move a change's head branch to the commit it landed as #542
  2. feat(runway): resolve merge targets per queue and provision their checkouts #544
  3. feat(orchestrator): build extension profiles from configuration #545
  4. @ test(e2e): hermetic end-to-end coverage of a real git merge #546
  5. feat(gateway): land and status subcommands for the client #547
  6. docs(submitqueue): provider example configuration and landing runbook #548

## Summary

### Why?

The existing E2E suite runs Runway on the noop merger, so a request reaching `landed` proves the pipeline's choreography and nothing at all about git. Every risky part of merging for real — the `git` binary in the image, checkout provisioning, the apply and push, head-branch updates — had no automated coverage through the stack.

None of that needs a provider. The merger speaks `git://` against any remote, so pointing it at a bare repository on a shared volume exercises the whole merge path with no credential, no network, and no account anywhere — which is what lets these assertions gate a pull request.

### What?

A second suite lands `git://` changes and asserts against the repository itself rather than against request status: which commits reached the target, in what order, and in how many ref updates. A three-change stack must arrive in exactly **one** ref update, counted from the target's reflog — the property that distinguishes a submit queue from merging changes one at a time, and one that is hard to check against a live provider.

It also pins two behaviors worth not regressing: a conflicting change leaves the target untouched, and re-submitting a change after it has landed is rejected as stale (landing moved its head branch, so the URI no longer describes where that branch points).

`testutil.NewComposeStack` gained `WithOverlay`, so the suite states only what differs from the base stack — Runway's merge target — instead of copying the whole service definition. A copy is the thing that rots: services added to the base file never reach the duplicate, and nothing fails until the variant runs.

The suite drives the bare repository with the same pinned git the merger uses, rather than the host's, since these assertions depend on repository mechanics and an ambient git brings ambient configuration with it.

## Test Plan

✅ `make e2e-git-test` — the new suite.

✅ `make e2e-test` — both suites; the existing one is unchanged and still passes.

Also fixes `make help`, whose target regex excluded digits, so neither `e2e-test` nor `e2e-git-test` was ever listed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant