Skip to content

agentcore run ab-test outputs incorrect invocation URL (uses runtime name instead of gateway target name) #1854

Description

@rhbelson

Description

When running agentcore run ab-test, the CLI outputs an invocationUrl that uses the runtime name (PascalCase, e.g. CustomerSupportAB) as the path segment, but the gateway target is actually registered with a different name (kebab-case, e.g. customer-support-ab). This causes all requests to the printed URL to fail with "No Target found for Target name: CustomerSupportAB".

Steps to Reproduce

  1. Create an agent runtime named CustomerSupportAB with a gateway target
  2. Run agentcore run ab-test --mode config-bundle --name cs_prompt_abtest --gateway my-gateway-secure --runtime
    CustomerSupportAB --control-bundle customerSupportControl --control-version LATEST --treatment-bundle
    customerSupportTreatment --treatment-version LATEST --online-eval ABQualityMonitor --control-weight 80 --treatment-weight 20
  3. Observe the printed invocation URL
  4. Use the printed URL with curl to send traffic
  5. Also try: agentcore view ab-test --json | jq -r '.invocationUrl'

Expected Behavior

The invocation URL should use the actual gateway target name. Based on agentcore status --json, the target is registered as customer-support-ab, so the correct URL is:

https://.gateway.bedrock-agentcore.us-west-2.amazonaws.com/customer-support-ab/invocations

Actual Behavior

The CLI prints and stores in .invocationUrl:

https://.gateway.bedrock-agentcore.us-west-2.amazonaws.com/CustomerSupportAB/invocations

Requests to this URL fail:
{"success":false,"error":"No Target found for Target name: CustomerSupportAB"}

The correct target name (customer-support-ab) is visible in agentcore status --json under
.deployedState.targets.default.resources.gateways["my-gateway-secure"].targets but the AB test command does not use it.

CLI Version

No response

Operating System

Linux

Additional Context

Workaround: build the URL manually from agentcore status --json using the gateway URL + the actual target key name:

GW_BASE=$(agentcore status --json | jq -r '.deployedState.targets.default.resources.gateways."my-gateway-secure".gatewayUrl')
GATEWAY_URL="${GW_BASE}/customer-support-ab/invocations"

This confirms the A/B test infrastructure itself works correctly — only the URL reported by the CLI is wrong. The mismatch
appears to be that agentcore run ab-test substitutes the --runtime argument value directly into the URL path, rather than
looking up the corresponding gateway target name.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions