Skip to content

feat(ci): run v1 canary against the published npm package - #2699

Merged
nborges-aws merged 2 commits into
refactorfrom
resolve-canary-from-npm
Oct 23, 2026
Merged

nborges-aws merged 2 commits into
refactorfrom
resolve-canary-from-npm

Conversation

@nborges-aws

Copy link
Copy Markdown
Contributor

Description

Change v1 canary to install and invoke the CLI from the npm vended @aws/agentcore@latest, rather than resolving from the local code. This ensures the canary will catch failure points that may only exist in the distributed artifact, and not the source code. This gives us closer visibility and experience to our customers installing and invoking the AgentCore CLI.

The matching Git tag still supplies the E2E test suite.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

How have you tested the change?

  • I ran bun test
  • I ran the relevant end-to-end tests with bun run test:e2e, or explained why they are not applicable
  • I ran bun run typecheck
  • I ran bun run lint:check
  • I ran bun run format:check
  • I ran bun run build
  • If I modified src/assets/, I updated affected snapshots with bun test <test-file> --update-snapshots and committed them

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

@nborges-aws
nborges-aws requested a review from a team October 23, 2026 18:57
@github-actions github-actions Bot added the size/s PR size: S label Oct 23, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Package Tarball

aws-agentcore-1.0.0.tgz

How to install

gh release download pr-2699-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-1.0.0.tgz

@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 23, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

I verified the change against the PR's base branch (refactor) and found no blocking issues.

What I checked:

  • canary.yml: version is assigned earlier in the same run block, so the new version=${version} output is populated correctly, and the pinned exact version removes any race with a publish happening between resolve_release and e2e.
  • Expression safety for workflow_dispatch: npmVersion is only declared under workflow_call, so on manual dispatch inputs.npmVersion is null. GitHub coerces both null and '' to 0 for loose comparison, so inputs.npmVersion == '' is true → Build CLI still runs and AGENTCORE_CLI_PATH falls back to node <workspace>/dist/index.js. No accidental agentcore path on dispatch.
  • Harness compatibility: e2eTest/helpers/run.ts splits AGENTCORE_CLI_PATH on whitespace and spawns it, so the bare agentcore value resolves via PATH on Linux. The published package's bin is agentcore → ./dist/index.js, which matches. Nothing under e2eTest/ imports from src/ or references dist/, so skipping bun run build doesn't break the suite.
  • No shadowing risk: the repo isn't a workspace and neither npm nor bun links a root package's own bin into node_modules/.bin, so bun run test:e2e's PATH prefix won't shadow the globally installed CLI with a stale/missing local shim.
  • Tag/suite compatibility: v1.0.0 has test:e2e and the canary tag, so the tagged suite + published-package combination lines up.
  • npm install --global is consistent with the existing canary-v0.yml pattern, and the version flows through an env var (quoted), so no injection concern.

Non-blocking notes for your consideration:

  1. npmVersion isn't exposed on workflow_dispatch (.github/workflows/e2e-test.yml, dispatch input block ~lines 22-36). That makes the new published-package path unverifiable by manual dispatch — the only way to exercise it is to wait for the scheduled canary. Also, the ref description was updated for workflow_call but the workflow_dispatch copy still reads "to build and test".
  2. AGENTCORE_CLI_PATH: agentcore won't work on Windows. The global npm shim is agentcore.cmd, and spawn() in e2eTest/helpers/run.ts runs without a shell, so Windows CreateProcess won't resolve it (ENOENT). This is currently masked because canary.yml passes linuxOnly: true, but it's a latent trap for any future caller that sets npmVersion with the full OS matrix. Options: resolve and pass the absolute path to the installed bin (e.g. npm prefix -g), document/guard npmVersion as Linux-only, or teach run.ts to use shell: true on win32.
  3. Minor hardening: agentcore --version is printed but not asserted against NPM_VERSION; comparing them would make a wrong-version install fail loudly instead of silently testing the wrong artifact.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Oct 23, 2026
@codecov-commenter

codecov-commenter commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.62%. Comparing base (94d9031) to head (b0f30c3).

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #2699   +/-   ##
=========================================
  Coverage     96.62%   96.62%           
=========================================
  Files           706      706           
  Lines         58447    58447           
=========================================
  Hits          56472    56472           
  Misses         1975     1975           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jariy17
jariy17 previously approved these changes Oct 23, 2026
@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Oct 23, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 23, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 23, 2026
@nborges-aws
nborges-aws merged commit 9ab9ed9 into refactor Oct 23, 2026
22 checks passed
@nborges-aws
nborges-aws deleted the resolve-canary-from-npm branch October 23, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants