[aws] Enable Identity Federation for aws.securityhub data stream and standalone SecurityHub integration by seanrathier · Pull Request #20529 · elastic/integrations

seanrathier · 2026-08-04T18:56:26Z

Proposed commit message

Enable Identity Federation (Cloud Connectors) for the SecurityHub data streams in the aws package and the standalone aws_securityhub package.

aws package (securityhub_findings, securityhub_findings_full_posture, securityhub_insights)

Migrates the three SecurityHub HTTPJSON stream templates from hand-rolled SigV4 signing (X-Amz-Date/Authorization set transforms, including pagination re-sign) to the input's auth.aws configuration with use_cloud_connectors driven by supports_identity_federation. Removes the identity_federation gate from the securityhub policy template. Fixes silent session-token breakage — the hand-rolled path never set X-Amz-Security-Token.

aws_securityhub standalone package

Adds var_groups with the full credential selector (Identity Federation, Direct Access Keys, Temporary Access Keys, Assume Role, Shared Credentials), bumps format_version to 3.6.4 and kibana/agent floors to ^9.6.0, removes external_id per ingest-dev#9116, and fixes pipeline hygiene violations surfaced by the format_version bump.

Part of elastic/ingest-dev#8812. Supersedes #20436.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs. (E2E validation pending — see test plan below)
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines
I have verified that any added dashboard complies with Kibana's Dashboard good practices (no dashboards added or changed)

Author's Checklist

E2E: all three aws SecurityHub streams validated via Identity Federation (HEALTHY, no AccessDeniedException)
E2E: aws_securityhub CEL stream validated via Identity Federation
[CFT] Add ElasticAwsSecurityHub policy to federated-identity-aws template cloudbeat#8030 merged and CFT published to S3 before this ships (adds securityhub:GetFindings + securityhub:GetInsights to the federated role)

How to test this PR locally

# aws package
cd packages/aws
elastic-package check
elastic-package test policy

# aws_securityhub package
cd packages/aws_securityhub
elastic-package check
elastic-package test policy

For the federated path E2E:

Deploy [CFT] Add ElasticAwsSecurityHub policy to federated-identity-aws template cloudbeat#8030's CFT (adds securityhub:GetFindings + securityhub:GetInsights to the federated role)
In Fleet, create an agentless AWS integration using the Identity Federation credential type
Enable the SecurityHub CSPM policy template (aws package) and validate securityhub_findings, securityhub_findings_full_posture, and securityhub_insights all show HEALTHY
Enable the aws_securityhub integration and validate the CEL stream shows HEALTHY

Related issues

Relates elastic/ingest-dev#8812
Relates elastic/ingest-dev#9116 (external_id removal)
Companion CFT PR: [CFT] Add ElasticAwsSecurityHub policy to federated-identity-aws template cloudbeat#8030
Supersedes [aws_securityhub] Enable Identity Federation for agentless deployments #20436
Relates [aws] Enable Identity Federation for all remaining policy templates #20527 (rest of the aws package federation)

github-actions · 2026-08-04T18:59:34Z

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

…t-dev#8812) Grants securityhub:GetFindings and securityhub:GetInsights to the ElasticFederatedIdentityRole, covering: - aws/securityhub_findings and aws/securityhub_findings_full_posture (POST /findings) - aws/securityhub_insights (POST /insights/get) - aws_securityhub/finding (POST /findingsv2 — uses securityhub:GetFindings IAM action) Mirrors the provider_permissions declared in: - elastic/integrations#20529 (aws package SecurityHub streams) - elastic/integrations#20436 (aws_securityhub package) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Enables Identity Federation (Cloud Connectors) for the aws package’s SecurityHub CSPM HTTPJSON data streams by migrating away from hand-rolled SigV4 signing to auth.aws, and lifts the policy-template gate that previously hid Identity Federation for SecurityHub. The PR also includes related aws_securityhub package updates (version/constraints, Identity Federation var group, and ingest pipeline tagging/lint compliance).

Changes:

Migrate securityhub_findings, securityhub_findings_full_posture, and securityhub_insights HTTPJSON templates to auth.aws (including optional use_cloud_connectors) and remove custom SigV4 header transforms.
Remove the SecurityHub policy template’s hide_in_var_group_options gate for identity_federation, bump aws package version to 7.3.0, and add changelog entry.
Update aws_securityhub package for Identity Federation/agentless support (constraints + var_groups), and apply ingest pipeline processor tagging / improved on_failure messaging.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/aws/manifest.yml	Bumps package version to 7.3.0 and removes the Identity Federation hide gate for the SecurityHub CSPM policy template.
packages/aws/changelog.yml	Adds 7.3.0 changelog entry describing the SecurityHub Identity Federation enablement/migration.
packages/aws/data_stream/securityhub_findings/agent/stream/httpjson.yml.hbs	Removes hand-rolled SigV4 transforms and configures `auth.aws` (incl. session token + optional cloud connectors).
packages/aws/data_stream/securityhub_findings_full_posture/agent/stream/httpjson.yml.hbs	Same `auth.aws` migration for the full-posture findings stream, plus minor formatting cleanup.
packages/aws/data_stream/securityhub_insights/agent/stream/httpjson.yml.hbs	Same `auth.aws` migration for insights stream and removes pagination-time re-signing transforms.
packages/aws_securityhub/manifest.yml	Updates constraints/version, adds credential var group with Identity Federation option, removes external_id var, and adds provider permissions info.
packages/aws_securityhub/changelog.yml	Adds entries for Identity Federation agentless auth support and ingest pipeline tagging/lint compliance.
packages/aws_securityhub/data_stream/finding/agent/stream/cel.yml.hbs	Adds `auth.aws.use_cloud_connectors` rendering controlled by `supports_identity_federation`.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/default.yml	Adds processor tags broadly, improves `on_failure` messages to include `_ingest.pipeline`, and fixes a mismatched pipeline tag label.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_actor.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_attack.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_device.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_evidence.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_finding.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_malware.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_metadata.yml	Adds tags to processors and updates `on_failure` message formatting; also removes stray whitespace in one field line.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_osint.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_resources.yml	Adds tags to processors and updates `on_failure` message formatting.
packages/aws_securityhub/data_stream/finding/elasticsearch/ingest_pipeline/pipeline_object_vulnerabilities.yml	Adds tags to processors and updates `on_failure` message formatting.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…late Grants the IAM permissions required by the aws/securityhub and aws_securityhub integrations for Federated Identity (Cloud Connectors) deployments. securityhub:GetFindings authorizes both POST /findings and POST /findingsv2 — both SecurityHub API operations share this single IAM action. securityhub:GetInsights authorizes POST /insights/get. Part of elastic/ingest-dev#8812. Paired with elastic/integrations#20529. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

packages/aws_securityhub/changelog.yml:6

The 1.3.0 changelog entry links to PR #20436, but this PR description says it supersedes #20436 and the actual changes are landing here. Update the changelog link to point at the PR that introduces the 1.3.0 release notes (this PR) so the changelog doesn’t reference a superseded/closed PR.

- version: "1.3.0"
  changes:
    - description: Enable Identity Federation (Cloud Connectors) authentication for agentless deployments.
      type: enhancement
      link: https://github.com/elastic/integrations/pull/20436

elastic-vault-github-plugin-prod · 2026-08-19T23:32:52Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

…0529 The 1.3.0 entry was pointing at the superseded PR. The changelog-link CI check requires the link match the open PR number. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…-dev#8812) Migrates securityhub_findings, securityhub_findings_full_posture, and securityhub_insights HTTPJSON templates from hand-rolled SigV4 signing to the input's auth.aws configuration. Removes the identity_federation gate from the securityhub policy template. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…nal_id from stream templates - Add securityhub:GetFindings and securityhub:GetInsights to the SecurityHub policy template input's provider_permissions declaration - Remove {{#if external_id}} blocks from securityhub_findings, securityhub_findings_full_posture, and securityhub_insights hbs templates (external_id was removed from the package in elastic#20527) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (1)

packages/aws/manifest.yml:972

provider_permissions was added under the SecurityHub input, but the package format_version remains 3.6.1. Please confirm the package-spec version used by elastic-package supports provider_permissions at 3.6.1; if not, bump format_version accordingly (and address any newly-triggered validators) so the package continues to validate/build.

        provider_permissions:
          - provider: aws
            description: Security Hub read access for findings and insights collection.
            permissions:
              - name: securityhub:GetFindings

Add a var_groups credential selector to the aws_securityhub package, offering Identity Federation (Cloud Connectors) on the agentless path while preserving every existing credential method for agent-based deployments. The CloudFormation template is rendered by the IaC Provider at onboard time from the declared provider_permissions; no static template URL is used. - Bump format_version to 3.6.4 (var_groups needs 3.6.0, provider_permissions needs 3.6.4) - Raise Kibana and Agent floors to ^9.4.0 (auth.aws cloud connector support in the CEL input requires Agent 9.4.0+) - Add supports_identity_federation var and the use_cloud_connectors hook in the CEL stream template - Declare provider_permissions on the cel input: the GetFindingsV2 API authorizes via securityhub:GetFindings per the AWS API reference Depends on the pipeline hygiene pre-landing in elastic#20435. Part of elastic/ingest-dev#8812. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Kibana renders the template via the IaC Provider first and falls back to opening this quick-create URL when the render fails (422/502). Same S3 bucket and URL shape as the aws package's existing URL, but pointing at the incremental federated-identity-aws template, version pinned to the package's Kibana floor minor. The fallback only functions once elastic/cloudbeat#7422 merges and publishes the template to S3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ingest-dev#9116) - Remove external_id from identity_federation var_group option vars - Remove assume_role_external_id var_group option entirely - Remove external_id input var declaration - Remove {{#if external_id}} block from CEL stream template - Update iac_template_url to 9.6.0 and drop stale RESOURCE_ID param Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…tors requires 9.6.0+) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

format_version 3.6.4 is required for provider_permissions, which was added to the SecurityHub HTTPJSON input in this PR. The version bump reflects the new 7.4.0 changelog entry (7.3.0 is already taken by the processor-tags PR elastic#20572 that merged to main concurrently). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

packages/aws/data_stream/securityhub_findings/agent/stream/httpjson.yml.hbs:92

There are no policy rendering tests covering the new Identity Federation path for this HTTPJSON template (e.g., asserting that auth.aws.use_cloud_connectors is rendered when supports_identity_federation: true). Other AWS HTTPJSON streams (for example GuardDuty) have _dev/test/policy fixtures for this, so adding similar fixtures for the SecurityHub streams would help prevent regressions in signing/auth behavior.

{{#if supports_identity_federation}}
  use_cloud_connectors: {{supports_identity_federation}}
{{/if}}

…ll_posture, insights) Add elastic-package test policy fixtures for the three SecurityHub HTTPJSON data streams (securityhub_findings, securityhub_findings_full_posture, securityhub_insights), covering both legacy credentials and agentless Identity Federation (cloud connector) cases. These tests lock in the auth.aws field rendering (including use_cloud_connectors) introduced by this PR, and serve as a regression guard for future template changes. Modelled after the existing guardduty policy test fixtures. Note: .expected files were hand-authored from the templates. If CI shows a diff, run: elastic-package stack up -d && elastic-package test policy --generate Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.

…l on_failure messages All 11 aws_securityhub ingest pipeline files had their pipeline-level on_failure error.message missing the required '_ingest.pipeline' reference (SVR00009). The messages used the multi-line block scalar format: {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{...}}}' Fixed to: {{{/_ingest.on_failure_processor_tag}}}in pipeline '{{{ _ingest.pipeline }}}' failed with message '{{{...}}}' The processor-level on_failure blocks were also updated in the same pass: they used '{{{_ingest.on_failure_pipeline}}}' which was replaced with '{{{_ingest.pipeline}}}' as required by the linter. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

…s filtering Credential vars (access_key_id, secret_access_key, session_token, shared_credential_file, credential_profile_name, role_arn, supports_identity_federation, assume_role_duration, assume_role_expiry_window) were at the CEL input level with show_user: true for the first three, placing them outside the reach of package-level var_groups filtering (shouldShowVar only applies to package-level vars). Move all credential vars to the package level with show_user: false so that shouldShowVar correctly hides vars not belonging to the selected credential_type option. proxy_url and ssl remain at the input level as they are connection/transport settings, not credentials. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

elastic-vault-github-plugin-prod · 2026-08-20T21:22:27Z

✅ All changelog entries have the correct PR link.

Copilot

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

infra-vault-gh-plugin-prod · 2026-08-20T23:51:45Z

💚 Build Succeeded

Buildkite Build
Commit: ea8f22c

History

💚 Build #48297 succeeded 9ace394
💔 Build #48295 failed 156e0ba
💔 Build #48291 failed 9881295
💔 Build #48228 failed f2ba4c6
💔 Build #48202 failed a4df94c
💔 Build #48194 failed 3c375d4

seanrathier mentioned this pull request Aug 4, 2026

[aws] Enable Identity Federation for all remaining policy templates #20527

Merged

10 tasks

seanrathier added the enhancement New feature or request label Aug 4, 2026

seanrathier mentioned this pull request Aug 19, 2026

[CFT] Add ElasticAwsSecurityHub policy to federated-identity-aws template elastic/cloudbeat#8030

Merged

seanrathier force-pushed the aws/securityhub-identity-federation branch from b494235 to 32d7bff Compare August 19, 2026 17:23

seanrathier mentioned this pull request Aug 19, 2026

[aws_securityhub] Enable Identity Federation for agentless deployments #20436

Closed

3 tasks

seanrathier force-pushed the aws/securityhub-identity-federation branch from 3c375d4 to a4df94c Compare August 19, 2026 18:43

seanrathier added Integration:aws AWS Integration:aws_securityhub AWS Security Hub labels Aug 19, 2026

seanrathier marked this pull request as ready for review August 19, 2026 18:49

Copilot AI lite review requested due to automatic review settings August 19, 2026 18:49

seanrathier requested review from a team as code owners August 19, 2026 18:49

Copilot started reviewing on behalf of seanrathier August 19, 2026 18:50 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 19, 2026 21:03

seanrathier force-pushed the aws/securityhub-identity-federation branch from 25436f8 to a4df94c Compare August 19, 2026 21:03

Copilot started reviewing on behalf of seanrathier August 19, 2026 21:03 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

Comment thread packages/aws_securityhub/changelog.yml Outdated

Copilot AI review requested due to automatic review settings August 19, 2026 21:09

Copilot started reviewing on behalf of seanrathier August 19, 2026 21:09 View session

Copilot AI reviewed Aug 19, 2026

View reviewed changes

Comment thread packages/aws/manifest.yml

Copilot AI review requested due to automatic review settings August 20, 2026 15:37

Copilot AI reviewed Aug 20, 2026

View reviewed changes

seanrathier and others added 5 commits August 20, 2026 11:44

[aws_securityhub] Bump kibana/agent floor to ^9.6.0 (use_cloud_connec…

f04a41c

…tors requires 9.6.0+) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings August 20, 2026 15:45

seanrathier force-pushed the aws/securityhub-identity-federation branch from 9881295 to 9cd7972 Compare August 20, 2026 15:45

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Comment thread packages/aws_securityhub/data_stream/finding/agent/stream/cel.yml.hbs

Copilot AI review requested due to automatic review settings August 20, 2026 16:01

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 20, 2026 17:18

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Copilot started reviewing on behalf of seanrathier August 20, 2026 17:57 View session

Copilot started reviewing on behalf of seanrathier August 20, 2026 18:07 View session

Copilot started reviewing on behalf of seanrathier August 20, 2026 18:35 View session

Copilot started reviewing on behalf of seanrathier August 20, 2026 20:10 View session

Copilot AI review requested due to automatic review settings August 20, 2026 21:18

Copilot started reviewing on behalf of seanrathier August 20, 2026 21:19 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

seanrathier changed the title ~~[aws] Enable Identity Federation for SecurityHub data streams~~ [aws] Enable Identity Federation for aws.securityhub data stream and standalone SecurityHub integration Aug 21, 2026

Uh oh!

Conversation

seanrathier commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Uh oh!

github-actions Bot commented Aug 4, 2026

✅ Elastic Docs Style Checker (Vale)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod Bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

elastic-vault-github-plugin-prod Bot commented Aug 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

infra-vault-gh-plugin-prod Bot commented Aug 20, 2026

💚 Build Succeeded

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

seanrathier commented Aug 4, 2026 •

edited

Loading

elastic-vault-github-plugin-prod Bot commented Aug 19, 2026 •

edited

Loading