Skip to content

ci: split fast checks from slow e2e apply - #131

Merged
chianyungcode merged 6 commits into
mainfrom
ci/split-fast-checks-e2e
Nov 3, 2026
Merged

chianyungcode merged 6 commits into
mainfrom
ci/split-fast-checks-e2e

Conversation

@chianyungcode

@chianyungcode chianyungcode commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Why

CI took ~13 min per PR, and every macOS run since 2026-10-16 had failed.

  • Nearly all of the time went to chezmoi apply, which installs ~100 Homebrew formulae on macOS (~13 min) and apt packages on Ubuntu (~3.5 min). It ran on every PR, even theme or keybinding changes.
  • macOS failed at the very end because herdr-plus isn't a Homebrew formula. It's already installed as a herdr plugin via cli-tool-setup.toml.
  • The render matrix ran once per OS even though it already simulates macOS internally.

Changes

  • ci.yaml runs on every PR and push to main, on Ubuntu only:
    • the full tests/ suite (16 tests), with zsh, shellcheck and shfmt from apt plus pinned fish 4 and jj releases.
    • new tests/chezmoi-package-names.sh: checks every Homebrew formula and cask against the formulae.brew.sh API, every tap entry against its GitHub repo, and every APT package against the apt cache, without installing anything.
  • e2e.yaml runs a real chezmoi apply on Ubuntu and macOS. It runs only on pushes to main that touch install logic, weekly (Mon 02:17 UTC) to catch upstream package drift, or manually with gh workflow run e2e.yaml --ref <branch>.
  • Removes automated_test.yaml.
  • Drops the nonexistent herdr-plus formula from homebrew.toml.
  • Adds a README note about the package-name check.

Fixed tests

All existing tests were broken or never ran in CI. Every one still guards real behavior, so none were deleted.

  • chezmoi-render-scripts.sh, chezmoi-script-behavior.sh: read the machine's own chezmoi config, so .xdg was missing anywhere but the author's laptop. They now generate a config.
  • chezmoi-render-scripts.sh, chezmoi-script-behavior.sh: also picked up the runner's CI=true, which makes templates skip CI-gated work. They now unset it and set CI only for the cases that test CI behavior.
  • chezmoi-script-behavior.sh: assumed every imperative tool was disabled. It now checks that the real data renders exactly its enabled entries. It also added gzip to a restricted PATH, since GNU tar needs it on Linux.
  • television-git-channels.sh: exact-string checks broke after taplo aligned the = signs. Key/value checks now ignore whitespace.
  • television and zsh git tests: a global tag.gpgsign=true broke git tag in temporary repos. Signing is now disabled there.
  • fish-sk-git.sh: stubbed only macOS open, not Linux xdg-open.
  • zsh tests and docs/06-cli-integrations.md: updated for the 90_sk-git.sh → sk-git.sh and 99_skim.sh → skim.sh renames. Dropped a guard for fzf.sh.tmpl, which the fzf-git → sk-git migration removed on purpose.
  • Real bug (b37d3406): sk-git.sh never deferred its Ctrl-G keybindings until after deferred antidote plugins, although its test expected that from day one. It now uses zsh-defer, like skim.sh and zoxide.sh.

Testing

  • All 16 tests pass on the GitHub runner, and in a clean ubuntu:24.04 container with and without CI=true.
  • chezmoi-package-names.sh passed on this PR's first CI run, including the APT check on the real Ubuntu runner.
  • Both workflows pass actionlint, including shellcheck on the run: blocks.

🤖 Generated with Claude Code

chianyungcode and others added 6 commits November 3, 2026 22:59
herdr-plus is not in homebrew-core; it is already installed as a herdr
plugin via cli-tool-setup.toml. The bogus formula made every macOS CI
run fail after ~12 minutes of package installs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ci.yaml: runs on every push/PR (Ubuntu only, ~1 min): init data,
  render matrix, and a new package-name check against Homebrew API,
  tap repositories, and apt cache.
- e2e.yaml: real `chezmoi apply` on Ubuntu and macOS, only on main
  pushes touching install logic, weekly, or manual dispatch.
- Remove automated_test.yaml, which ran the ~13 min macOS install on
  every PR and duplicated the render matrix per OS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind the Ctrl-G sk-git widgets through zsh-defer when available, matching
skim.sh and zoxide.sh, so deferred antidote plugins cannot override them.
tests/zsh-sk-git.sh has expected this since the module was added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- render-scripts, script-behavior: generate a chezmoi config instead of
  reading the machine's own, so .xdg/.identity exist in clean
  environments like CI.
- script-behavior: assert the real imperative tool data renders exactly
  its enabled entries, instead of assuming every entry is disabled; add
  gzip to the restricted PATH so GNU tar can extract on Linux.
- television-git-channels: compare key/value lines ignoring taplo's
  `=` alignment.
- television and zsh git tests: disable tag signing in temporary repos.
- fish-sk-git: stub xdg-open, the opener used on Linux.
- zsh tests and docs: follow the 90_sk-git.sh -> sk-git.sh and
  99_skim.sh -> skim.sh renames; drop the guard for fzf.sh.tmpl, which
  was intentionally removed in the fzf-git -> sk-git migration.
- Mark all test scripts executable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Install zsh, shellcheck, shfmt, and pinned fish 4 / jj releases, then run
every test under tests/ with grouped logs and per-test error annotations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub runners export CI=true, which makes templates skip CI-gated
work (standalone tools, casks). The render-scripts and script-behavior
tests set CI explicitly for the cases that need it, so unset any
inherited value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chianyungcode
chianyungcode force-pushed the ci/split-fast-checks-e2e branch from 0e13b7f to ccbae77 Compare November 3, 2026 16:00
@chianyungcode
chianyungcode merged commit 0ea41a1 into main Nov 3, 2026
1 check passed
@chianyungcode
chianyungcode deleted the ci/split-fast-checks-e2e branch November 3, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant