From 051ebd2255a73a0fefd2ab5e179ab854dccc2412 Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:24:28 +0000 Subject: [PATCH 1/7] =?UTF-8?q?fix(CI-CD):=20=F0=9F=90=9B=20drop=20stale?= =?UTF-8?q?=20shellcheck=20exclusions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit path-keys.sh doesn't exist anymore (removed in dotfiles-sync's opt-in SSH keys refactor) and sync-files.sh now passes shellcheck clean (verified directly: shellcheck -S warning src/dotfiles-sync/sync-files.sh, no output). Only gh-auth.sh legitimately needs the exclusion — it's sourced from /etc/profile.d/ and can't carry a shebang (SC2148). Co-Authored-By: Claude Sonnet 5 --- .github/workflows/pr-validation.yml | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pr-validation.yml b/.github/workflows/pr-validation.yml index c3b864d..a47d093 100644 --- a/.github/workflows/pr-validation.yml +++ b/.github/workflows/pr-validation.yml @@ -160,6 +160,8 @@ jobs: baseImage: mcr.microsoft.com/devcontainers/base:debian - features: github-dev baseImage: mcr.microsoft.com/devcontainers/base:ubuntu + - features: copilot-dev + baseImage: mcr.microsoft.com/devcontainers/base:ubuntu - features: mistral-dev baseImage: ubuntu:latest - features: mistral-dev @@ -211,6 +213,8 @@ jobs: # exists. Add an entry here whenever a feature gains a # scenarios.json; see AGENTS.md "Testing non-default options". - features: claude-dev + - features: cline-dev + - features: mistral-dev steps: - name: Checkout @@ -233,9 +237,10 @@ jobs: with: severity: warning scandir: src - # Preserve the previous scope (install.sh only) — path-keys.sh, sync-files.sh, - # and gh-auth.sh have never been linted here and may have pre-existing issues. - ignore_names: "path-keys.sh sync-files.sh gh-auth.sh" + # gh-auth.sh is sourced from /etc/profile.d/, so it can't carry a + # shebang — shellcheck's SC2148 ("target shell unknown") on that is + # expected and not fixable without breaking how the file is used. + ignore_names: "gh-auth.sh" - name: Verify AGENTS.md's feature table matches manifest versions run: | From 7551cb57e92e428f16f84e0f4f9e5e97de8abecb Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:54:19 +0000 Subject: [PATCH 2/7] =?UTF-8?q?test(copilot-dev):=20=E2=9C=85=20add=20miss?= =?UTF-8?q?ing=20test=20coverage?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This feature had no test/ directory at all and no CI matrix entry in either workflow — the published feature had never once been exercised by CI. Adds a test.sh that asserts the documented behavior for both cases it can hit: the gh copilot extension installed when gh is present, or the graceful skip when it isn't (copilot-dev has no dependsOn on github-dev by design, so the latter is what a standalone install actually gets by default). No version bump — test-only, invisible to consumers. Co-Authored-By: Claude Sonnet 5 --- test/copilot-dev/test.sh | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 test/copilot-dev/test.sh diff --git a/test/copilot-dev/test.sh b/test/copilot-dev/test.sh new file mode 100755 index 0000000..4364917 --- /dev/null +++ b/test/copilot-dev/test.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash + +# This file is part of helpers4. +# Copyright (C) 2026 baxyz +# SPDX-License-Identifier: LGPL-3.0-or-later +# +# copilot-dev has no dependsOn on github-dev (AI assistant only, deliberately +# separate from platform tooling — see the README), so a standalone test has +# no gh CLI to install the extension against. That's the actual default +# path most consumers hit before adding github-dev themselves; assert it +# degrades the way the feature documents, not just "didn't crash". + +set -e + +echo "Testing copilot-dev feature..." + +if command -v gh >/dev/null 2>&1; then + if gh extension list 2>/dev/null | grep -qi copilot; then + echo "✅ PASS: gh copilot extension installed (gh CLI was present)" + else + echo "❌ FAIL: gh CLI present but gh copilot extension not installed" + exit 1 + fi +else + echo "✅ PASS: gh CLI absent — install.sh's documented graceful-skip path, as expected without github-dev" +fi + +echo "🎉 Test passed." From 7df2973a5285dbbb6d87f091c94d52048c1fdc2d Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:54:19 +0000 Subject: [PATCH 3/7] =?UTF-8?q?test(essential-dev):=20=E2=9C=85=20assert?= =?UTF-8?q?=20the=20git=20aliases=20test.sh=20silently=20ignored?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit test.sh was a no-op — two echo lines regardless of what install.sh actually did. The feature's one real, build-time artifact (git config --system alias.fixup/alias.polish) was never verified. Now asserts both aliases resolve to their expected values. No version bump — test-only, invisible to consumers. Co-Authored-By: Claude Sonnet 5 --- test/essential-dev/test.sh | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) mode change 100644 => 100755 test/essential-dev/test.sh diff --git a/test/essential-dev/test.sh b/test/essential-dev/test.sh old mode 100644 new mode 100755 index dab063e..4732627 --- a/test/essential-dev/test.sh +++ b/test/essential-dev/test.sh @@ -6,8 +6,22 @@ set -e echo "🧪 Testing essential-dev feature..." -# The feature is configuration-only (VS Code extensions) -# These tests verify the feature executes without errors +# The only real, build-time artifact this feature writes to disk — the VS +# Code extensions/settings in its manifest are applied by the devcontainer +# CLI itself, not by install.sh, so there's nothing else on the filesystem +# to assert here. +FIXUP="$(git config --system --get alias.fixup || true)" +if [ "${FIXUP}" != "commit --fixup" ]; then + echo "❌ FAIL: git alias.fixup is '${FIXUP}', expected 'commit --fixup'" + exit 1 +fi +echo "✅ PASS: git alias.fixup configured" + +POLISH="$(git config --system --get alias.polish || true)" +if [ "${POLISH}" != "rebase -i --autosquash origin/HEAD" ]; then + echo "❌ FAIL: git alias.polish is '${POLISH}', expected 'rebase -i --autosquash origin/HEAD'" + exit 1 +fi +echo "✅ PASS: git alias.polish configured" -echo "✅ Feature executed successfully" echo "✅ essential-dev tests passed" From c3015f2168db822abbe15159b884c15da11db975 Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:54:19 +0000 Subject: [PATCH 4/7] =?UTF-8?q?test(cline-dev):=20=E2=9C=85=20add=20an=20i?= =?UTF-8?q?nstallCli:true=20scenario?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Same gap class claude-dev just got a scenario for: install.sh already degrades gracefully (warns instead of aborting) on a failed npm install, which is correct — but installCli defaults to false and test.sh only ever asserted the CLI's *absence* under default options. The true-install path had zero CI coverage. Verified directly: devcontainer features test --features cline-dev --skip-autogenerated . installs cline 3.0.61 correctly. No version bump — test-only, invisible to consumers. Co-Authored-By: Claude Sonnet 5 --- test/cline-dev/scenarios.json | 10 ++++++++++ test/cline-dev/with_cli.sh | 28 ++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 test/cline-dev/scenarios.json create mode 100755 test/cline-dev/with_cli.sh diff --git a/test/cline-dev/scenarios.json b/test/cline-dev/scenarios.json new file mode 100644 index 0000000..65d68cc --- /dev/null +++ b/test/cline-dev/scenarios.json @@ -0,0 +1,10 @@ +{ + "with_cli": { + "image": "mcr.microsoft.com/devcontainers/typescript-node:22", + "features": { + "cline-dev": { + "installCli": true + } + } + } +} diff --git a/test/cline-dev/with_cli.sh b/test/cline-dev/with_cli.sh new file mode 100755 index 0000000..e83d22e --- /dev/null +++ b/test/cline-dev/with_cli.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash + +# This file is part of helpers4. +# Copyright (C) 2026 baxyz +# SPDX-License-Identifier: LGPL-3.0-or-later +# +# Exercises installCli:true — the default-off code path plain test.sh never +# runs. install.sh already degrades gracefully (warns instead of aborting) +# on a failed npm install, which is correct, but that means a real failure +# here would otherwise be silent — assert the CLI is actually there. + +set -e + +echo "Testing cline-dev with installCli:true..." + +if ! command -v cline >/dev/null 2>&1; then + echo "❌ FAIL: cline CLI not found — installCli:true did not install it" + exit 1 +fi +echo "✅ PASS: cline CLI on PATH" + +if ! cline --version >/dev/null 2>&1; then + echo "❌ FAIL: cline is on PATH but does not run" + exit 1 +fi +echo "✅ PASS: cline --version runs" + +echo "🎉 Test passed." From 643aae5ca37cce85767b395ab7406a4cf99d8962 Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:54:19 +0000 Subject: [PATCH 5/7] =?UTF-8?q?test(mistral-dev):=20=E2=9C=85=20add=20an?= =?UTF-8?q?=20installCli:true=20scenario?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Same gap class claude-dev just got a scenario for. install.sh's uv branch in particular trusts uv tool install without verifying the binary actually landed afterward — exactly the class of gap already found once in vite-plus — and installCli defaults to false with zero prior CI coverage of the true-install path. Verified directly: devcontainer features test --features mistral-dev --skip-autogenerated . installs vibe (via pip, Python 3.12) correctly. No version bump — test-only, invisible to consumers. Co-Authored-By: Claude Sonnet 5 --- test/mistral-dev/scenarios.json | 10 ++++++++++ test/mistral-dev/with_cli.sh | 28 ++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 test/mistral-dev/scenarios.json create mode 100755 test/mistral-dev/with_cli.sh diff --git a/test/mistral-dev/scenarios.json b/test/mistral-dev/scenarios.json new file mode 100644 index 0000000..ff6832d --- /dev/null +++ b/test/mistral-dev/scenarios.json @@ -0,0 +1,10 @@ +{ + "with_cli": { + "image": "mcr.microsoft.com/devcontainers/python:3.12", + "features": { + "mistral-dev": { + "installCli": true + } + } + } +} diff --git a/test/mistral-dev/with_cli.sh b/test/mistral-dev/with_cli.sh new file mode 100755 index 0000000..a522ee5 --- /dev/null +++ b/test/mistral-dev/with_cli.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash + +# This file is part of helpers4. +# Copyright (C) 2026 baxyz +# SPDX-License-Identifier: LGPL-3.0-or-later +# +# Exercises installCli:true — the default-off code path plain test.sh never +# runs. install.sh's uv branch in particular trusts `uv tool install` +# without verifying the binary actually landed afterward — assert it here +# rather than leaving that path unverified. + +set -e + +echo "Testing mistral-dev with installCli:true..." + +if ! command -v vibe >/dev/null 2>&1; then + echo "❌ FAIL: vibe CLI not found — installCli:true did not install it" + exit 1 +fi +echo "✅ PASS: vibe CLI on PATH" + +if ! vibe --version >/dev/null 2>&1; then + echo "❌ FAIL: vibe is on PATH but does not run" + exit 1 +fi +echo "✅ PASS: vibe --version runs" + +echo "🎉 Test passed." From cd239603bccf84cb3edf9443a6c389a463150691 Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:54:19 +0000 Subject: [PATCH 6/7] =?UTF-8?q?ci(CI-CD):=20=F0=9F=91=B7=20wire=20copilot-?= =?UTF-8?q?dev/cline-dev/mistral-dev=20into=20test.yml?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Counterpart to pr-validation.yml's matrix updates from the shellcheck-fix commit — this repo's push-to-main workflow needs the same entries. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/test.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1e1d720..57f9779 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -117,6 +117,11 @@ jobs: - features: github-dev baseImage: mcr.microsoft.com/devcontainers/base:ubuntu + # copilot-dev - no dependsOn on github-dev; gh CLI absent by + # default, install.sh's own graceful-skip path + - features: copilot-dev + baseImage: mcr.microsoft.com/devcontainers/base:ubuntu + # dotfiles-sync - works on any base image - features: dotfiles-sync baseImage: mcr.microsoft.com/devcontainers/base:debian @@ -166,6 +171,8 @@ jobs: # exists. Add an entry here whenever a feature gains a # scenarios.json; see AGENTS.md "Testing non-default options". - features: claude-dev + - features: cline-dev + - features: mistral-dev steps: - name: Checkout From e119823ab518bcd804a78de80e936e3b0ed033be Mon Sep 17 00:00:00 2001 From: baxyz Date: Wed, 9 Sep 2026 21:54:29 +0000 Subject: [PATCH 7/7] =?UTF-8?q?fix(CI-CD):=20=F0=9F=90=9B=20version-bump-c?= =?UTF-8?q?heck=20must=20never=20require=20a=20bump=20for=20test/=20change?= =?UTF-8?q?s?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects an over-broad rule from #97: test//test.sh itself was kept as a bump trigger there (reasoning: "the one file CI runs against every consumer's build"), only exempting scenario files. That's still wrong — test.sh is exactly as invisible to a consumer as a scenario file. A version bump publishes a new GHCR tag and triggers a website docs rebuild; doing that for a change nobody running the feature can see is pure noise, not a safety measure. Narrows the check to src// (minus README) only. Nothing under test// ever requires a bump anymore. Updated AGENTS.md's "Modifying an existing feature" and "Testing non-default options" sections to state this explicitly, and reverted the version bumps this branch had already added to copilot-dev and essential-dev for test-only changes. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/pr-validation.yml | 26 ++++++------- AGENTS.md | 58 +++++++++++++++-------------- 2 files changed, 42 insertions(+), 42 deletions(-) diff --git a/.github/workflows/pr-validation.yml b/.github/workflows/pr-validation.yml index a47d093..ef7dd08 100644 --- a/.github/workflows/pr-validation.yml +++ b/.github/workflows/pr-validation.yml @@ -38,9 +38,9 @@ jobs: echo "::notice::Diff base = ${MERGE_BASE} → HEAD ($(git rev-parse HEAD))" # Computed once, reused per-feature below instead of re-running git - # diff in the loop. Covers both src// (what ships) and - # test//test.sh — a sibling of src/, not nested under it, but - # AGENTS.md requires a bump for it too, so it needs its own pathspec. + # diff in the loop. test/ is included only so a test-only PR still + # resolves a feature name to check (and reports "no bump needed" + # correctly) — it never counts toward requiring a bump itself. touched_files=$(git diff --name-only "${MERGE_BASE}..HEAD" -- src/ test/) mapfile -t names < <(echo "$touched_files" | awk -F/ '{print $2}' | sort -u) @@ -53,22 +53,20 @@ jobs: new_version=$(jq -r '.version' "$manifest") old_version=$(git show "${MERGE_BASE}:${manifest}" 2>/dev/null | jq -r '.version' 2>/dev/null || echo "") - # A version bump is required for anything under src// except - # its README, or for test//test.sh specifically — the one - # test file CI runs against every consumer's build. A new - # test//scenarios.json or scenario script is test-only, like - # a README change: it verifies a non-default option combination - # but doesn't touch what already-published consumers get. - non_readme_touched=$({ \ - echo "$touched_files" | grep -E "^src/${name}/" | grep -v "^src/${name}/README\.md$"; \ - echo "$touched_files" | grep -E "^test/${name}/test\.sh$"; \ - } || true) + # A version bump is required for anything under src// + # except its README — nothing under test// counts, + # including test.sh itself. Tests and CI are invisible to the + # consumer: bumping for them publishes a new GHCR tag and + # triggers a docs rebuild for a change nobody running the + # feature can actually see. + non_readme_touched=$(echo "$touched_files" \ + | grep -E "^src/${name}/" | grep -v "^src/${name}/README\.md$" || true) if [ -z "$old_version" ]; then echo "✅ ${name}: new feature, no bump required" elif [ "$old_version" = "$new_version" ]; then if [ -z "$non_readme_touched" ]; then - echo "ℹ️ ${name}: README-only change, version bump optional (still ${new_version})" + echo "ℹ️ ${name}: README/test-only change, no bump needed (still ${new_version})" else echo "❌ ${name}: touched but version unchanged (${new_version})" FAILED=1 diff --git a/AGENTS.md b/AGENTS.md index 10444a8..bd51bfb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -51,10 +51,9 @@ re-run once per that job's `baseImage` matrix entries. Not every option needs a reserve it for an off-by-default path whose failure mode is silent (a warn-and-continue on install failure, not a hard `exit 1`) rather than one that already fails loudly if broken. -Adding a `scenarios.json` or scenario script doesn't need a version bump — like a README, it -verifies an existing option combination without changing what already-published consumers get. -Only `test//test.sh` itself (the one CI runs against every consumer's build) triggers the -version-bump-check below. +Adding a `scenarios.json`, a scenario script, or editing `test.sh` itself never needs a version +bump — none of it is visible to a consumer of the feature, see "Modifying an existing feature — +version bump" below. **Available features:** @@ -92,32 +91,35 @@ version-bump-check below. **Modifying an existing feature — version bump:** -Any change under `src//` that touches `install.sh`, -`devcontainer-feature.json`, or `test//test.sh` must bump that -feature's `version` field (patch by default, minor/major when warranted) — -`release.yml` only tags and publishes a feature whose `version` changed -between the base branch and HEAD, so an unbumped change to something that -actually ships silently never gets published. Bump it **once per branch**: -if the version on the branch already differs from `main`'s, a further commit -on that same branch/PR must *not* bump it again — check the diff against -`main` first, don't bump reflexively on every commit. +Any change under `src//` that touches `install.sh` or +`devcontainer-feature.json` must bump that feature's `version` field (patch +by default, minor/major when warranted) — `release.yml` only tags and +publishes a feature whose `version` changed between the base branch and +HEAD, so an unbumped change to something that actually ships silently +never gets published. Bump it **once per branch**: if the version on the +branch already differs from `main`'s, a further commit on that same +branch/PR must *not* bump it again — check the diff against `main` first, +don't bump reflexively on every commit. -A **README-only** change doesn't require a bump — nothing about what ships -in the image changes. It's still worth bumping when the doc fix is -safety-relevant (e.g. a corrected `initializeCommand` requirement, like -`dotfiles-sync` v1.0.8), since `release.yml`'s version-diff gate is also -what triggers the website docs rebuild — an unbumped README fix never -reaches the published site. Judgment call, not enforced either way. +**Never bump for something invisible to the consumer** — nothing under +`test//` (including `test.sh` itself and any scenario file), and no +CI workflow change, ever needs a bump. A version bump publishes a new GHCR +tag and triggers a website docs rebuild; doing that for a change nobody +running the feature can actually see is pure noise. A **README-only** +change doesn't require one either — nothing about what ships in the image +changes. It's still worth bumping a README fix when it's safety-relevant +(e.g. a corrected `initializeCommand` requirement, like `dotfiles-sync` +v1.0.8), since the docs-rebuild trigger is exactly what's needed there. +Judgment call, not enforced either way. -Enforced by the `version-bump-check` job in `pr-validation.yml`: it fails the -PR if a touched feature's `version` is unchanged from `main` *and* something -other than `src//README.md` changed under `src//`, or -`test//test.sh` specifically changed (a new `test//scenarios.json` -or scenario script doesn't count — see "Testing non-default options" above). -It's a blocking check only — it never commits a bump on your behalf (deliberately: -no bot commits, no push-permission/fork edge cases, consistent with how -`conventional-commits` already works in this repo). Bump the version -yourself and push again. +Enforced by the `version-bump-check` job in `pr-validation.yml`: it fails +the PR if a touched feature's `version` is unchanged from `main` *and* +something other than `src//README.md` changed under `src//`. +Nothing under `test//` ever counts toward this check — see "Testing +non-default options" above. It's a blocking check only — it never commits +a bump on your behalf (deliberately: no bot commits, no push-permission/ +fork edge cases, consistent with how `conventional-commits` already works +in this repo). Bump the version yourself and push again. **A lost version bump after merge gets flagged, not just caught before merge.** `version-bump-check` only validates the PR branch — it can't catch a bump