Skip to content

[aws_mq] Enable Identity Federation for agentless deployments - #20817

Open
seanrathier wants to merge 4 commits into
elastic:mainfrom
seanrathier:aws_mq/identity-federation
Open

[aws_mq] Enable Identity Federation for agentless deployments#20817
seanrathier wants to merge 4 commits into
elastic:mainfrom
seanrathier:aws_mq/identity-federation

Conversation

@seanrathier

@seanrathier seanrathier commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

Enable Identity Federation (Cloud Connectors) for the Amazon MQ integration.

Adds var_groups with a credential selector (Identity Federation, Direct Access Keys, Temporary Access Keys, Assume Role, Shared Credentials), enables agentless deployment mode, and wires use_cloud_connectors into all five stream templates. Bumps format_version to 3.6.4 and kibana/agent floors to ^9.6.0 (required for use_cloud_connectors). Fixes pipeline processor tag and on_failure hygiene surfaced by the format_version bump.

Major version bump (1.0.02.0.0) frees the 1.x namespace for backport-aws_mq-1.x, which will carry patch/minor fixes for users on stacks below ^9.6.0. See elastic/ingest-dev#8788 for the branching strategy.

Part of elastic/ingest-dev#8812.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs. (E2E validation pending)
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices (no dashboards added or changed)

Author's Checklist

  • backport-aws_mq-1.x branch created on elastic/integrations from the last 1.0.0 release commit
  • CODEOWNERS sign-off from @elastic/obs-infraobs-integrations on the major version bump + Kibana floor change
  • E2E: activemq_metrics and rabbitmq_metrics validated via Identity Federation
  • E2E: activemq_audit_logs, activemq_general_logs, rabbitmq_general_logs validated via Identity Federation
  • CFT permissions already covered by ElasticAwsMetrics and ElasticAwsCloudwatchLogs in [CFT] Add ElasticAwsSecurityHub policy to federated-identity-aws template cloudbeat#8030

Related

🤖 Generated with Claude Code

…-dev#8812)

- Bump format_version 3.3.1 → 3.6.4 (required for var_groups + provider_permissions)
- Bump version 1.0.0 → 2.0.0 (major bump frees 1.x namespace for backport-aws_mq-1.x)
- Bump kibana and agent floors to ^9.6.0 (use_cloud_connectors requires Agent 9.6.0+)
- Add supports_identity_federation var
- Add var_groups with identity_federation, direct_access_key, temporary_access_key,
  assume_role, and shared_credentials options
- Enable agentless deployment mode (beta) on the amazon_mq policy template
- Add use_cloud_connectors to all 5 stream templates (activemq_metrics,
  rabbitmq_metrics, activemq_audit_logs, activemq_general_logs, rabbitmq_general_logs)
- Fix pipeline processor tags and on_failure messages (SVR00006/SVR00009) surfaced
  by the format_version bump

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Elastic Docs Style Checker (Vale)

Summary: 1 suggestion found

💡 Suggestions (1): Optional style improvements. Apply when helpful.
File Line Rule Message
packages/aws_mq/changelog.yml 1 Elastic.Versions Use 'later versions' instead of 'newer versions' when referring to versions.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

Package aws_mq 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
activemq_audit_logs 12500 9523.81 -2976.19 (-23.81%) 💔

To see the full report comment with /test benchmark fullreport

Declares IAM permissions needed for Identity Federation role provisioning
via the IaCP render path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@seanrathier
seanrathier force-pushed the aws_mq/identity-federation branch from c0c0947 to 26c6004 Compare August 19, 2026 21:06
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@seanrathier
seanrathier marked this pull request as ready for review August 21, 2026 18:11
@seanrathier
seanrathier requested a review from a team as a code owner August 21, 2026 18:11
Copilot AI lite review requested due to automatic review settings August 21, 2026 18:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables Identity Federation and agentless deployment for the Amazon MQ integration.

Changes:

  • Adds credential selection, agentless support, permissions, and updated version floors.
  • Enables Cloud Connectors across all five stream templates.
  • Updates ingest pipelines and records the 2.0.0 release.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Summary
packages/aws_mq/manifest.yml Adds federation, agentless configuration, permissions, and version updates.
packages/aws_mq/data_stream/rabbitmq_metrics/elasticsearch/ingest_pipeline/default.yml Updates processor tags and failure context.
packages/aws_mq/data_stream/rabbitmq_metrics/agent/stream/stream.yml.hbs Enables Cloud Connectors.
packages/aws_mq/data_stream/rabbitmq_general_logs/elasticsearch/ingest_pipeline/default.yml Updates processor tags and failure context.
packages/aws_mq/data_stream/rabbitmq_general_logs/agent/stream/aws-cloudwatch.yml.hbs Enables Cloud Connectors.
packages/aws_mq/data_stream/activemq_metrics/elasticsearch/ingest_pipeline/default.yml Updates processor tags and failure context.
packages/aws_mq/data_stream/activemq_metrics/agent/stream/stream.yml.hbs Enables Cloud Connectors.
packages/aws_mq/data_stream/activemq_general_logs/elasticsearch/ingest_pipeline/default.yml Updates processor tags and failure context.
packages/aws_mq/data_stream/activemq_general_logs/agent/stream/aws-cloudwatch.yml.hbs Enables Cloud Connectors.
packages/aws_mq/data_stream/activemq_audit_logs/elasticsearch/ingest_pipeline/default.yml Updates processor tags and failure context.
packages/aws_mq/data_stream/activemq_audit_logs/agent/stream/aws-cloudwatch.yml.hbs Enables Cloud Connectors.
packages/aws_mq/changelog.yml Documents the 2.0.0 release.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…reaking-change

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 24, 2026 19:17
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

packages/aws_mq/manifest.yml:154

  • The shared aws/metrics input performs ec2:DescribeRegions during startup (the AWS package documents it as a required call), and the federated ElasticAwsMetrics policy grants it. This new provider_permissions declaration omits that action, so the Amazon MQ metrics input under-declares the permissions needed for Identity Federation. Please add ec2:DescribeRegions here.
              - name: cloudwatch:GetMetricData
              - name: cloudwatch:ListMetrics
              - name: tag:GetResources

packages/aws_mq/manifest.yml:140

  • Enabling the agentless deployment mode without regenerating the package documentation leaves both docs/README.md and _dev/build/docs/README.md saying that Elastic Agent must be installed, with no Agentless deployment section. That directly contradicts this new mode and will mislead users; rerun the package build (or update the generated docs) and include the resulting documentation changes.
      agentless:
        enabled: true
        release: beta

Comment on lines +138 to +140
agentless:
enabled: true
release: beta

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seanrathier , do you want to take up this suggestion ?

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

History

@agithomas agithomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good to me. Kindly address the suggestion for improvement.

@mergify

mergify Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants