feat(scorecard): introduce scorecard dora plugin

[dzemanov]

Hey, I just made a Pull Request!

Introduces new backstage-plugin-scorecard-backend-module-dora.

Incorporates PRs:

• Scorecard GitHub DORA: feat(scorecard): Scorecard DORA GitHub dzemanov/rhdh-plugins#6
• Scorecard Jira DORA: feat(scorecard): Scorecard DORA Jira dzemanov/rhdh-plugins#7
• Scorecard timeseries API: feat(scorecard-dora): Timeseries API dzemanov/rhdh-plugins#10

Additional changes:

• incorporated changes from main branch for scorecard after rebase
• moved jira incidents collector issueType app-config configuration under dora collector input in e7c248f, so configuration looks like this:

scorecard:
  metricProviders:
    dora:
      meanTimeToRestore:
        options:
          collectors:
            incidents:
              id: jira:incidents
              # Optional: override default Incident issue type
              # input:
              #   issueType: ServiceIncident

Fixes

Fixes https://redhat.atlassian.net/browse/RHIDP-13982
Incorporates https://redhat.atlassian.net/browse/RHIDP-13978, https://redhat.atlassian.net/browse/RHIDP-13979, https://redhat.atlassian.net/browse/RHIDP-13980

DORA Metrics

• dora.deploymentFrequency
• dora.medianLeadTimeForChanges
• dora.changeFailureRate
• dora.meanTimeToRestore

dora.deploymentFrequency

• Uses deployments from last 30 days.
• Counts only successful deployments.
• Counts production deployments (or deployments with unknown environment).
• Returns normalized value in deployments/week
• Result: (successfulProductionDeployments / 30) * 7 (to deployments/week)

Thresholds deploymentFrequency

Updated (medium to 1-7 to cover the whole real line):

Elite (≥7/week)
Medium (1-7/week)
Low (<1/week)

dora.medianLeadTimeForChanges

• Uses deployments from last 30 days in chronological order.
• Counts only successful deployments.
• Counts production deployments (or deployments with unknown environment).
• Iterates deployment pairs (previous -> current).
• Resolves PRs for commit range between these deployment pairs (baseCommitSha -> headCommitSha).
• For each PR: leadTimeHours = deployedAt - firstCommitAt.
• Returns median lead time in hours
• Result: MEDIAN for (deployedAtTimestamp - firstCommitAtTimestamp) / 3_600_000 (to hours)

Thresholds medianLeadTimeForChanges

Updated to use single unit (h):

Elite <24
Medium 24-168
Low >168

dora.changeFailureRate

• Uses deployments from last 30 days.
• Counts only successful deployments.
• Counts production deployments (or deployments with unknown environment).
• Evaluates adjacent deployment intervals: [deployment.createdAt, nextDeployment.createdAt)
• Marks starting deployment of each interval as failed if at least one incident falls into interval
• Returns mean of recovery hours
• Result: (failedDeployments / successfulDeployments) * 100

Thresholds changeFailureRate

Elite (<5%)
Medium (5-15%)
Low (>15%)

dora.meanTimeToRestore

• Uses incidents from the last 30 days
• Considers resolved incidents (resolutionDate != null)
• Computes recovery time per incident: resolutionDate - createdAt (in hours)
• Returns median recovery time in hours
• Result: MEAN for (incidentResolutionAtTimestamp - incidentCreatedAtTimestamp) / 3_600_000 (to hours)

Thresholds meanTimeToRestore

Elite (<1h)
Medium (1-24h)
Low (>24h)

DORA collectors

DORA metrics are composite metrics that require data from different third parties for their computation. They use Scorecard Collectors, reusable components designed to gather data from various datasources, such as Jira or GitHub. Users can create their custom data collector to tailor data collection for DORA metrics calculation for their specific setup.

• github:deployments — Fetches GitHub Deployments in a time window from GH Deployments API
• github:deploymentWorkflowRuns — Fetches Deployments from GitHub Actions - treats matching GitHub Actions workflow runs as deployments
• github:deploymentPullRequests — Fetches PRs (and first-commit times) between two deployment SHAs
• jira:incidents — Fetches Jira incident issues in a time window

Scorecard Jira changes

• extracts annotation and JQL building into standalone functions as now we require it for both jira.opneIssues and jira:incidents collector
• adds paginated request support to Cloud and DataCenter jira clients

Demo

How to test

Incorporated PRs have been tested, you can skip to case 7 foradditional changes this PR introduces.

Navigate to scorecard-dora component.

1. Test out configuration without changes (default collectors, you need to comment out dora in app-config.yaml)
2. Test out explicit default collectors:

scorecard:
  plugins:
    dora:
      deploymentFrequency:
        options:
          collectors:
            deployments:
              id: github:deployments
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }
      medianLeadTimeForChanges:
        options:
          collectors:
            deployments:
              id: github:deployments
            deploymentPullRequests:
              id: github:deploymentPullRequests
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }
      changeFailureRate:
        options:
          collectors:
            deployments:
              id: github:deployments
            incidents:
              id: jira:incidents
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }
      meanTimeToRestore:
        options:
          collectors:
            incidents:
              id: jira:incidents
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }

3. Test out workflow runs collector:

scorecard:
  plugins:
    dora:
      deploymentFrequency:
        options:
          collectors:
            deployments:
              id: github:deploymentWorkflowRuns
              input:
                workflowName: Create Test Deployment on PR Merge
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }
      medianLeadTimeForChanges:
        options:
          collectors:
            deployments:
              id: github:deploymentWorkflowRuns
              input:
                workflowName: Create Test Deployment on PR Merge
            deploymentPullRequests:
              id: github:deploymentPullRequests
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }
      changeFailureRate:
        options:
          collectors:
            deployments:
              id: github:deploymentWorkflowRuns
              input:
                workflowName: Create Test Deployment on PR Merge
            incidents:
              id: jira:incidents
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 10 }

4. Verify entity annotations for incidents collector
   For default jira:incidents, entity should have:

jira/incident-project-key (preferred), or
jira/project-key (fallback)

Other annotations:
jira/incident-component: Component
jira/incident-label: UI
jira/incident-team: 9d3ea319-fb5b-4621-9dab-05fe502283e
jira/incident-issue-type: MyIncident

You can log what jql filters are applied here.

5. Verify productionEnvironments options

scorecard:
  metricProviders:
    dora:
      deploymentFrequency:
        options:
          productionEnvironments:
            - production
            - prod
            - test-deployment
      medianLeadTimeForChanges:
        options:
          productionEnvironments:
            - production
            - prod
      changeFailureRate:
        options:
          productionEnvironments:
            - production
            - prod

6. Verify time-series API
   Refer to How to test section in feat(scorecard-dora): Timeseries API dzemanov/rhdh-plugins#10

7. Verify app-config options for incidents collector
   For example changeFailureRate:

      changeFailureRate:
        options:
          collectors:
            deployments:
              id: github:deployments
            incidents:
              id: jira:incidents
              input:
                issueType: CustomIncident

You can log what jql filters are applied here.

✔️ Checklist

• A changeset describing the change and affected packages. (more info)
• Added or Updated documentation
• Tests for new functionality and regression tests for bug fixes
• Screenshots attached (for UI changes)

[rhdh-gh-app]

Important

This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
backend	workspaces/scorecard/packages/backend	none	v0.0.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dora	workspaces/scorecard/plugins/scorecard-backend-module-dora	minor	v0.0.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-github	workspaces/scorecard/plugins/scorecard-backend-module-github	minor	v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-jira	workspaces/scorecard/plugins/scorecard-backend-module-jira	minor	v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend	workspaces/scorecard/plugins/scorecard-backend	minor	v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard-common	workspaces/scorecard/plugins/scorecard-common	minor	v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard	workspaces/scorecard/plugins/scorecard	minor	v4.2.0

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 5:35 PM UTC · Ended 5:40 PM UTC

Commit: fbdaafd · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:40 PM UTC · Completed 6:00 PM UTC

Commit: 1f75596 · View workflow run →

[rhdh-qodo-merge]

PR Summary by Qodo

Add DORA scorecard backend module with collectors and time-series API

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Add a new Scorecard backend module providing four DORA metrics with defaults and history.
• Introduce GitHub/Jira collectors needed for DORA calculations (deployments, workflow runs, PRs,
 incidents).
• Add a metric time-series endpoint backed by indexed DB queries and API/schema updates.

Diagram

graph TD
  A["Scorecard Backend"] --> B["DORA Module"] --> C["Collectors Service"]
  C --> D["GitHub Module"] --> E{{"GitHub API"}}
  C --> F["Jira Module"] --> G{{"Jira API"}}
  A --> H[("metric_values DB")]

  subgraph Legend
    direction LR
    _svc(["Service/Module"]) ~~~ _db[("Database")] ~~~ _ext{{"External"}}
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Pre-aggregate daily series at write time

• ➕ Time-series reads become O(days) without scanning raw rows
• ➕ Simplifies server-side folding logic (latest sample per UTC day)
• ➖ More complex write pipeline (extra table or upsert logic)
• ➖ Harder to change bucketing logic later without backfill

2. Leverage existing DORA provider/collector library

• ➕ Less custom code to maintain
• ➕ Potentially broader datasource support out of the box
• ➖ Hard to align with Scorecard collector contracts and permission model
• ➖ External dependencies may not match Backstage/RHDH conventions

3. Compute DORA purely from stored scorecard history (no live collectors)

• ➕ No external API calls during metric calculation
• ➕ More deterministic and replayable metrics
• ➖ Requires separate ingestion pipeline to populate deployments/incidents/PR data
• ➖ Loses flexibility for per-entity dynamic data fetch based on annotations

Recommendation: Current approach (Scorecard-native collectors + providers, plus a dedicated time-series read API) is the best fit for the existing plugin architecture and extension points. Keep the runtime calculation model, but consider pre-aggregating daily samples later if DB growth or time-series latency becomes an issue.

Files changed (83) +7430 / -67

Enhancement (32) +2447 / -36

quick-monkeys-wash.mdChangeset for DORA module, collectors framework, and time-series API +16/-0

Changeset for DORA module, collectors framework, and time-series API

• Introduces a multi-package minor changeset describing the new DORA backend module, new collectors, a time-series API endpoint, and a new metric visualization hint.

workspaces/scorecard/.changeset/quick-monkeys-wash.md

index.tsRegister the DORA backend module in the backend app +5/-0

Register the DORA backend module in the backend app

• Adds backend.add(import('@...scorecard-backend-module-dora')) to enable DORA providers in the example backend.

workspaces/scorecard/packages/backend/src/index.ts

constants.tsDORA defaults (collector IDs, time window, prod envs) +22/-0

DORA defaults (collector IDs, time window, prod envs)

• Defines default collector IDs, 30-day time window, and default production environment list used across providers.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/constants.ts

index.tsExport DORA backend module entrypoint +7/-7

Export DORA backend module entrypoint

• Replaces placeholder content with package documentation and exports the default backend module feature.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/index.ts

module.tsRegister DORA metric providers in Scorecard backend +60/-0

Register DORA metric providers in Scorecard backend

• Creates a backend module (pluginId scorecard/moduleId dora) that registers four DORA metric providers using the collectors service and (where needed) logger/config.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/module.ts

DoraConfig.tsCentralized DORA config parsing and default thresholds +271/-0

Centralized DORA config parsing and default thresholds

• Adds config parsing helpers for collector wiring and production environment handling. Defines default threshold rules for all four DORA metrics.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraConfig.ts

DoraDeploymentFrequencyProvider.tsDORA deployment frequency provider +132/-0

DORA deployment frequency provider

• Implements deployments/week over a 30-day window using a deployments collector, production-environment filtering, and sparkline default visualization.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.ts

DoraMedianLeadTimeForChangesProvider.tsDORA median lead time for changes provider +206/-0

DORA median lead time for changes provider

• Computes lead-time hours from PR first-commit time to deployment time across adjacent production deployments. Uses PR collection for commit ranges and returns a median value.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts

DoraChangeFailureRateProvider.tsDORA change failure rate provider +195/-0

DORA change failure rate provider

• Calculates percent of deployment intervals containing incidents using deployments+incidents collectors over a 30-day window; returns a percentage with sparkline visualization default.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraChangeFailureRateProvider.ts

DoraMeanTimeToRestoreProvider.tsDORA mean time to restore provider +162/-0

DORA mean time to restore provider

• Computes mean recovery time (hours) for resolved incidents, skipping invalid timestamps and returning a sparkline-friendly metric.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMeanTimeToRestoreProvider.ts

deploymentSchemas.tsZod schemas for deployment collector contracts +63/-0

Zod schemas for deployment collector contracts

• Defines deployment input/output schemas used by DORA providers to validate collector contracts.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/schemas/deploymentSchemas.ts

incidentSchemas.tsZod schemas for incident collector contracts +38/-0

Zod schemas for incident collector contracts

• Defines incident input/output schemas (including resolutionAt) used by DORA incident-based providers.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/schemas/incidentSchemas.ts

pullRequestSchemas.tsZod schemas for deployment PR collector contracts +42/-0

Zod schemas for deployment PR collector contracts

• Defines input/output schemas for collecting PRs between deployment commit SHAs.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/schemas/pullRequestSchemas.ts

calculationUtils.tsShared mean/median utilities for DORA providers +34/-0

Shared mean/median utilities for DORA providers

• Adds reusable calculation helpers for central tendency calculations.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/utils/calculationUtils.ts

deploymentFilterUtils.tsDeployment production/success filtering utilities +49/-0

Deployment production/success filtering utilities

• Adds shared helpers to treat missing environment as production and filter to successful production deployments (case-insensitive env matching).

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/utils/deploymentFilterUtils.ts

GithubDeploymentsCollector.tsNew GitHub deployments collector +114/-0

New GitHub deployments collector

• Adds a collector that fetches GitHub deployments in a time window and maps them into the Scorecard deployment schema.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentsCollector.ts

GithubDeploymentWorkflowRunsCollector.tsNew workflow-run based deployments collector +127/-0

New workflow-run based deployments collector

• Adds an alternative collector that derives deployments from GitHub Actions workflow runs in a time window.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentWorkflowRunsCollector.ts

GithubDeploymentPullRequestsCollector.tsNew collector for PRs between deployments +132/-0

New collector for PRs between deployments

• Collects commit SHAs between two deployment commits and resolves linked PRs, returning deduped PRs with firstCommitAt required for lead time metrics.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentPullRequestsCollector.ts

deploymentsSchemas.tsShared schema for deployment collector outputs +33/-0

Shared schema for deployment collector outputs

• Defines zod schemas used by GitHub deployment collectors to validate their output payloads.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/schemas/deploymentsSchemas.ts

GithubClient.tsExpand GitHub client for deployments/workflow runs/commit range PRs +312/-7

Expand GitHub client for deployments/workflow runs/commit range PRs

• Refactors the client to accept a logger, centralizes credentials provider usage, adds REST Octokit support, and introduces GraphQL queries for deployments and commit-linked PRs with improved error handling.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/github/GithubClient.ts

module.tsRegister GitHub collectors via collectors extension point +18/-3

Register GitHub collectors via collectors extension point

• Adds scorecardCollectorsExtensionPoint usage to register the new GitHub collectors, and updates provider wiring to pass logger into GithubOpenPRsProvider.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/module.ts

JiraIncidentsCollector.tsNew Jira incidents collector +90/-0

New Jira incidents collector

• Adds a collector that builds incident JQL from entity annotations, supports issueType override, and returns issues as incident records.

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/collectors/JiraIncidentsCollector.ts

incidentJql.tsIncident JQL builder with issueType resolution +65/-0

Incident JQL builder with issueType resolution

• Implements JQL generation for incidents with safe value validation/sanitization and defaults with annotation override precedence.

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/collectors/incidentJql.ts

jiraIncidents.tsDefault incident issue type constant +17/-0

Default incident issue type constant

• Introduces DEFAULT_INCIDENT_ISSUE_TYPE used when no config or annotation override is present.

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/constants/jiraIncidents.ts

module.tsRegister Jira incidents collector and refactor provider wiring +16/-3

Register Jira incidents collector and refactor provider wiring

• Registers JiraIncidentsCollector via collectors extension point and constructs a shared Jira client via JiraClientFactory for providers/collectors.

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/module.ts

router.tsAdd time-series route for entity metrics +29/-0

Add time-series route for entity metrics

• Adds GET /metrics/catalog/:kind/:namespace/:name/time-series with query validation, permission checks, entity access checks, and delegation to CatalogMetricService time-series API.

workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts

CatalogMetricService.tsImplement getEntityMetricTimeSeries service method +88/-1

Implement getEntityMetricTimeSeries service method

• Adds a service method that validates entity existence, enforces metric authorization, reads metric_values rows in range, folds to latest sample per UTC day, and returns a typed MetricTimeSeriesResponse.

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts

DatabaseMetricValues.tsAdd DB query for metric values in timestamp range +24/-0

Add DB query for metric values in timestamp range

• Adds readEntityMetricValuesInRange to fetch ordered rows for a specific entity+metric between timestamps, used by the time-series service.

workspaces/scorecard/plugins/scorecard-backend/src/database/DatabaseMetricValues.ts

collector.tsIntroduce CollectorConfig type in scorecard-common +25/-0

Introduce CollectorConfig type in scorecard-common

• Adds a public CollectorConfig type (id + optional JSON input) for configuring collectors in app-config and provider configs.

workspaces/scorecard/plugins/scorecard-common/src/types/collector.ts

Metric.tsAdd defaultVisualization and time-series response types +37/-0

Add defaultVisualization and time-series response types

• Extends Metric metadata with defaultVisualization and adds MetricTimeSeriesPoint/MetricTimeSeriesResponse types for the new API contract.

workspaces/scorecard/plugins/scorecard-common/src/types/Metric.ts

index.tsExport new common types (collector/time-series) +1/-0

Export new common types (collector/time-series)

• Updates type exports so CollectorConfig and time-series types are available to consumers.

workspaces/scorecard/plugins/scorecard-common/src/types/index.ts

CardWrapper.tsxShow full metric description on hover +17/-15

Show full metric description on hover

• Wraps truncated description text in a Tooltip so the full description is accessible without expanding the card layout.

workspaces/scorecard/plugins/scorecard/src/components/Common/CardWrapper.tsx

Bug fix (1) +44 / -0

validateTimeSeriesQueryParams.tsValidate time-series query parameters +44/-0

Validate time-series query parameters

• Adds a zod-based validator enforcing required metricId/from/to and from<=to, returning InputError on invalid requests.

workspaces/scorecard/plugins/scorecard-backend/src/middlewares/validateTimeSeriesQueryParams.ts

Refactor (1) +10 / -6

GithubOpenPRsProvider.tsInject logger-backed GitHub client into open PRs provider +10/-6

Inject logger-backed GitHub client into open PRs provider

• Refactors provider construction to accept a GithubClient instance and updates fromConfig signature to include logger.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/GithubOpenPRsProvider.ts

Tests (21) +3456 / -3

DoraConfig.test.tsTests for DORA configuration parsing and defaults +250/-0

Tests for DORA configuration parsing and defaults

• Adds unit tests validating config parsing behavior, fallbacks, and default threshold structures.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraConfig.test.ts

DoraDeploymentFrequencyProvider.test.tsTests for deployment frequency provider +236/-0

Tests for deployment frequency provider

• Adds tests covering deployment filtering and normalization logic for deployments/week calculations.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.test.ts

DoraMedianLeadTimeForChangesProvider.test.tsTests for median lead time provider +444/-0

Tests for median lead time provider

• Adds comprehensive unit tests for interval iteration, PR handling, and median calculation behavior including error cases.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.test.ts

DoraChangeFailureRateProvider.test.tsTests for change failure rate provider +434/-0

Tests for change failure rate provider

• Adds tests for interval evaluation, incident detection, and edge cases (insufficient deployments / no evaluable intervals).

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraChangeFailureRateProvider.test.ts

DoraMeanTimeToRestoreProvider.test.tsTests for mean time to restore provider +239/-0

Tests for mean time to restore provider

• Adds tests for resolved-incident filtering, invalid data handling, and mean calculation.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMeanTimeToRestoreProvider.test.ts

index.tsDORA provider test fixtures index +19/-0

DORA provider test fixtures index

• Adds fixture exports to simplify tests across DORA providers.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/fixtures/index.ts

mockCollectors.tsMock collector outputs for DORA tests +84/-0

Mock collector outputs for DORA tests

• Provides mock deployments/incidents/PRs datasets used in provider unit tests.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/fixtures/mockCollectors.ts

mockCollectorsService.tsMock collectors service for provider tests +48/-0

Mock collectors service for provider tests

• Adds a fake ScorecardCollectorsService implementation for deterministic test execution.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/fixtures/mockCollectorsService.ts

mockEntity.tsMock catalog entity for DORA tests +27/-0

Mock catalog entity for DORA tests

• Adds representative entity fixtures used across provider tests.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/fixtures/mockEntity.ts

calculationUtils.test.tsTests for calculation utilities +53/-0

Tests for calculation utilities

• Adds unit tests validating mean/median helpers.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/utils/calculationUtils.test.ts

deploymentFilterUtils.test.tsTests for deployment filter utilities +61/-0

Tests for deployment filter utilities

• Validates environment matching and success filtering behavior used by all deployment-based DORA metrics.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/utils/deploymentFilterUtils.test.ts

GithubDeploymentsCollector.test.tsTests for GitHub deployments collector +124/-0

Tests for GitHub deployments collector

• Adds unit tests validating input validation and mapping semantics for deployments collection.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentsCollector.test.ts

GithubDeploymentWorkflowRunsCollector.test.tsTests for workflow runs collector +124/-0

Tests for workflow runs collector

• Adds unit tests validating workflow run mapping to deployment-like records.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentWorkflowRunsCollector.test.ts

GithubDeploymentPullRequestsCollector.test.tsTests for deployment pull request collector +114/-0

Tests for deployment pull request collector

• Adds tests ensuring PR dedupe and missing firstCommitAt handling.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentPullRequestsCollector.test.ts

GitHubClient.test.tsUpdate GitHub client tests for new APIs +777/-1

Update GitHub client tests for new APIs

• Extends/updates unit tests to cover deployments, workflow runs, commit range retrieval, and repository-not-found handling.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/github/GitHubClient.test.ts

GithubOpenPRsProvider.test.tsUpdate open PRs provider tests for new construction API +9/-2

Update open PRs provider tests for new construction API

• Updates tests to account for provider creation changes and logger injection.

workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/GithubOpenPRsProvider.test.ts

router.test.tsAdd route tests for metric time-series endpoint +137/-0

Add route tests for metric time-series endpoint

• Adds tests for authorization, entity access checks, parameter validation errors, and happy-path responses for the new time-series route.

workspaces/scorecard/plugins/scorecard-backend/src/service/router.test.ts

plugin.api.test.tsIntegration test coverage for time-series API +29/-0

Integration test coverage for time-series API

• Adds end-to-end API tests verifying empty time-series behavior and 404 for missing entities.

workspaces/scorecard/plugins/scorecard-backend/src/plugin.api.test.ts

validateQueryAndParams.test.tsUnit tests for time-series query validator +59/-0

Unit tests for time-series query validator

• Adds validator tests for required params, ISO datetime validation, and from/to ordering semantics.

workspaces/scorecard/plugins/scorecard-backend/src/middlewares/validateQueryAndParams.test.ts

DatabaseMetricValues.test.tsTests for readEntityMetricValuesInRange query semantics +171/-0

Tests for readEntityMetricValuesInRange query semantics

• Adds cross-database tests ensuring ordering, filtering by entity/metric, inclusivity of bounds, and multiple samples per day behavior.

workspaces/scorecard/plugins/scorecard-backend/src/database/DatabaseMetricValues.test.ts

CardWrapper.test.tsxAdd/adjust CardWrapper tooltip tests +17/-0

Add/adjust CardWrapper tooltip tests

• Updates tests to validate tooltip behavior for truncated card descriptions.

workspaces/scorecard/plugins/scorecard/src/components/Common/tests/CardWrapper.test.tsx

Documentation (18) +1120 / -4

dora-scorecard.yamlAdd example component annotated for DORA +15/-0

Add example component annotated for DORA

• Adds a sample catalog Component with GitHub project slug/source-location, scorecard.io/dora enablement annotation, and Jira incident project annotation.

workspaces/scorecard/examples/components/dora-scorecard.yaml

README.mdDocumentation for installing and configuring DORA module +163/-0

Documentation for installing and configuring DORA module

• Documents prerequisites, installation, required entity annotation, available metrics, and threshold customization patterns for the DORA backend module.

workspaces/scorecard/plugins/scorecard-backend-module-dora/README.md

change-failure-rate.mdChange failure rate metric documentation +199/-0

Change failure rate metric documentation

• Adds detailed metric definition, data requirements, collector expectations, and examples for DORA change failure rate.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/change-failure-rate.md

deployment-frequency.mdDeployment frequency metric documentation +136/-0

Deployment frequency metric documentation

• Adds detailed metric definition, deployment filtering rules, and configuration guidance for DORA deployment frequency.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/deployment-frequency.md

mean-time-to-restore.mdMean time to restore metric documentation +114/-0

Mean time to restore metric documentation

• Documents incident requirements and MTTR calculation behavior, including handling of resolved incidents.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/mean-time-to-restore.md

median-lead-time-for-changes.mdMedian lead time for changes metric documentation +193/-0

Median lead time for changes metric documentation

• Documents lead time calculation based on deployments and PR commit times, plus configuration/collector requirements.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/median-lead-time-for-changes.md

report.api.mdAPI extractor report for DORA module package +11/-0

API extractor report for DORA module package

• Adds API report indicating the exported default BackendFeature for the DORA module.

workspaces/scorecard/plugins/scorecard-backend-module-dora/report.api.md

README.mdDocument GitHub collectors for DORA support +66/-0

Document GitHub collectors for DORA support

• Updates GitHub module documentation to cover new collectors used for DORA (deployments, workflow runs, deployment PRs).

workspaces/scorecard/plugins/scorecard-backend-module-github/README.md

README.mdDocument Jira incident collector and configuration +69/-4

Document Jira incident collector and configuration

• Updates Jira module docs to cover incident collection and DORA-related configuration patterns.

workspaces/scorecard/plugins/scorecard-backend-module-jira/README.md

report.api.mdAPI extractor report updated for new types +32/-0

API extractor report updated for new types

• Updates generated API report to include CollectorConfig, defaultVisualization, and metric time-series types.

workspaces/scorecard/plugins/scorecard-common/report.api.md

providers.mdDocument defaultVisualization usage for metrics +2/-0

Document defaultVisualization usage for metrics

• Adds guidance for using defaultVisualization (value vs sparkline) when defining metrics intended for time-series UI.

workspaces/scorecard/plugins/scorecard-backend/docs/providers.md

report.api.mdUpdate scorecard frontend API report for DORA translations +11/-0

Update scorecard frontend API report for DORA translations

• Adds translation keys for DORA metric titles/descriptions and new threshold labels (elite/medium/low) to the generated API report.

workspaces/scorecard/plugins/scorecard/report.api.md

de.tsAdd German translations for DORA metrics and thresholds +17/-0

Add German translations for DORA metrics and thresholds

• Adds localized strings for DORA metric labels/descriptions and elite/medium/low thresholds.

workspaces/scorecard/plugins/scorecard/src/translations/de.ts

es.tsAdd Spanish translations for DORA metrics and thresholds +18/-0

Add Spanish translations for DORA metrics and thresholds

• Adds localized strings for DORA metric labels/descriptions and elite/medium/low thresholds.

workspaces/scorecard/plugins/scorecard/src/translations/es.ts

fr.tsAdd French translations for DORA metrics and thresholds +18/-0

Add French translations for DORA metrics and thresholds

• Adds localized strings for DORA metric labels/descriptions and elite/medium/low thresholds.

workspaces/scorecard/plugins/scorecard/src/translations/fr.ts

it.tsAdd Italian translations for DORA metrics and thresholds +17/-0

Add Italian translations for DORA metrics and thresholds

• Adds localized strings for DORA metric labels/descriptions and elite/medium/low thresholds.

workspaces/scorecard/plugins/scorecard/src/translations/it.ts

ja.tsAdd Japanese translations for DORA metrics and thresholds +16/-0

Add Japanese translations for DORA metrics and thresholds

• Adds localized strings for DORA metric labels/descriptions and elite/medium/low thresholds.

workspaces/scorecard/plugins/scorecard/src/translations/ja.ts

ref.tsAdd reference translations for DORA metrics and thresholds +23/-0

Add reference translations for DORA metrics and thresholds

• Adds canonical/reference strings for DORA metric labels/descriptions and elite/medium/low thresholds.

workspaces/scorecard/plugins/scorecard/src/translations/ref.ts

Other (10) +353 / -18

app-config.yamlAdd example DORA metric provider configuration +59/-0

Add example DORA metric provider configuration

• Adds config blocks for DORA metric providers, wiring deployments/incidents collectors and schedules. Documents optional workflow-run based deployments and incident issueType override under collector input.

workspaces/scorecard/app-config.yaml

all-scorecards-location.yamlUpdate scorecard examples location list +1/-0

Update scorecard examples location list

• Adjusts the examples location file to include the new DORA example component definition.

workspaces/scorecard/examples/all-scorecards-location.yaml

package.jsonInclude DORA backend module dependency in example backend workspace +1/-0

Include DORA backend module dependency in example backend workspace

• Adds the new scorecard-backend-module-dora workspace package dependency so the example backend can load it.

workspaces/scorecard/packages/backend/package.json

.eslintrc.jsESLint configuration for new DORA module package +1/-0

ESLint configuration for new DORA module package

• Adds package-specific lint configuration to align with repository standards.

workspaces/scorecard/plugins/scorecard-backend-module-dora/.eslintrc.js

config.d.tsConfig schema for DORA metric providers +99/-0

Config schema for DORA metric providers

• Defines typed configuration for DORA providers including productionEnvironments, collector wiring, thresholds, and schedules.

workspaces/scorecard/plugins/scorecard-backend-module-dora/config.d.ts

package.jsonNew DORA backend module package manifest +71/-0

New DORA backend module package manifest

• Introduces the @red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dora package with Backstage module metadata, config schema, dependencies, and build scripts.

workspaces/scorecard/plugins/scorecard-backend-module-dora/package.json

package.jsonBump GitHub backend module package metadata for new collectors +3/-1

Bump GitHub backend module package metadata for new collectors

• Updates package dependencies/metadata to support newly added collectors and updated GitHub client usage.

workspaces/scorecard/plugins/scorecard-backend-module-github/package.json

package.jsonUpdate Jira backend module dependencies for new collector/client changes +4/-2

Update Jira backend module dependencies for new collector/client changes

• Adjusts dependencies/metadata to support the incidents collector and JiraClientFactory refactor usage.

workspaces/scorecard/plugins/scorecard-backend-module-jira/package.json

20260804123239_add_entity_metric_timestamp_index.jsIndex metric_values for time-series range reads +33/-0

Index metric_values for time-series range reads

• Adds an index on (catalog_entity_ref, metric_id, timestamp) to support efficient time-series queries; includes down migration.

workspaces/scorecard/plugins/scorecard-backend/migrations/20260804123239_add_entity_metric_timestamp_index.js

yarn.lockLockfile updates for new packages/dependencies +81/-15

Lockfile updates for new packages/dependencies

• Updates yarn.lock to reflect the newly added DORA module package and dependency changes in github/jira/scorecard packages.

workspaces/scorecard/yarn.lock

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Unsafe ID numeric compare 🐞 Bug ≡ Correctness

Description

CatalogMetricService.getEntityMetricTimeSeries chooses the “latest per day” row by comparing
Number(row.id), which can lose precision for bigint IDs and select the wrong sample. Since
readEntityMetricValuesInRange already returns rows ordered by timestamp asc then id asc, the numeric
comparison is unnecessary and can be replaced with “last row wins” (or a BigInt-safe comparison).

Code

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts[R239-242]

+      const existing = latestByUtcDay.get(dayKey);
+      // Postgres may return bigIncrements as strings; compare numerically.
+      if (!existing || Number(row.id) > Number(existing.id)) {
+        latestByUtcDay.set(dayKey, row);

Relevance

●●● Strong

BigInt-unsafe Number coercion is a clear correctness bug; team commonly accepts small logic-safety
fixes.

PR-#2913
PR-#2393

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The service currently relies on Number(row.id) to pick the latest row per UTC day, but the DB
query already guarantees ordering by timestamp and id ascending. This makes the numeric coercion
unnecessary and potentially incorrect for large bigint IDs due to JS Number precision limits.

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts[233-244]
workspaces/scorecard/plugins/scorecard-backend/src/database/DatabaseMetricValues.ts[231-253]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`CatalogMetricService.getEntityMetricTimeSeries` compares `row.id` values by coercing them to `Number(...)`. For Postgres `bigint` IDs, this becomes unsafe once values exceed `Number.MAX_SAFE_INTEGER` and is also redundant because the DB query already orders rows by `timestamp` then `id` ascending.

## Issue Context
The time-series endpoint reads rows via `DatabaseMetricValues.readEntityMetricValuesInRange`, which applies `.orderBy([{ timestamp asc }, { id asc }])`. Iterating that ordered result means that for any given UTC day, the last row encountered is already the latest by `(timestamp, id)`.

## Fix Focus Areas
- workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts[233-244]
- workspaces/scorecard/plugins/scorecard-backend/src/database/DatabaseMetricValues.ts[231-253]

## Suggested change
- Replace the `Number(row.id) > Number(existing.id)` comparison with a simple unconditional `latestByUtcDay.set(dayKey, row)` (since iteration is ordered), or use a BigInt-safe compare if you want to keep the explicit guard.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context

✅ Compliance rules (platform): 11 rules

✅ Cross-repo context

  Explored: repo: redhat-developer/rhdh (sha: 820e6260)

  Explored: repo: redhat-developer/rhdh-operator (sha: a425373c)

  Explored: repo: redhat-developer/rhdh-local (sha: a1776caa)

  Not relevant to this PR: redhat-developer/rhdh-chart

Tip of the day

💡 Did you know, you can type 'qodo, fix this' on a finding and the fix lands right on your PR

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[codecov]

Codecov Report

❌ Patch coverage is 91.75377% with 71 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.34%. Comparing base (71104e8) to head (ad926c2).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4235      +/-   ##
==========================================
- Coverage   59.59%   59.34%   -0.25%     
==========================================
  Files        2458     2489      +31     
  Lines       98268    98836     +568     
  Branches    27469    27474       +5     
==========================================
+ Hits        58559    58656      +97     
- Misses      39344    39898     +554     
+ Partials      365      282      -83     

Flag	Coverage Δ		*Carryforward flag
adoption-insights	84.55% <ø> (ø)		Carriedforward from 62d5e86
ai-integrations	68.29% <ø> (ø)		Carriedforward from 62d5e86
app-defaults	69.79% <ø> (ø)		Carriedforward from 62d5e86
augment	46.67% <ø> (ø)		Carriedforward from 62d5e86
boost	77.63% <ø> (ø)		Carriedforward from 62d5e86
bulk-import	72.79% <ø> (ø)		Carriedforward from 62d5e86
cost-management	13.55% <ø> (ø)		Carriedforward from 62d5e86
dcm	67.21% <ø> (ø)		Carriedforward from 62d5e86
e2e-adoption-insights	60.00% <ø> (ø)		Carriedforward from 62d5e86
e2e-extensions	?
e2e-intelligent-assistant	46.74% <ø> (ø)		Carriedforward from 62d5e86
extensions	56.59% <ø> (ø)		Carriedforward from 62d5e86
global-floating-action-button	71.18% <ø> (ø)		Carriedforward from 62d5e86
global-header	66.50% <ø> (ø)		Carriedforward from 62d5e86
homepage	47.50% <ø> (ø)		Carriedforward from 62d5e86
install-dynamic-plugins	59.95% <ø> (ø)		Carriedforward from 62d5e86
intelligent-assistant	75.42% <ø> (ø)		Carriedforward from 62d5e86
konflux	91.98% <ø> (ø)		Carriedforward from 62d5e86
lightspeed	69.02% <ø> (ø)		Carriedforward from 62d5e86
mcp-integrations	83.40% <ø> (ø)		Carriedforward from 62d5e86
orchestrator	71.31% <ø> (ø)		Carriedforward from 62d5e86
quickstart	63.74% <ø> (ø)		Carriedforward from 62d5e86
sandbox	79.56% <ø> (ø)		Carriedforward from 62d5e86
scorecard	87.06% <91.75%> (+0.81%)	⬆️
theme	88.14% <ø> (ø)		Carriedforward from 62d5e86
translations	5.12% <ø> (ø)		Carriedforward from 62d5e86
x2a	79.20% <ø> (ø)		Carriedforward from 62d5e86

*This pull request uses carry forward flags. Click here to find out more.

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 71104e8...ad926c2. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[fullsend-ai-review]

Review

Findings

Medium

• [scope-creep] workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts:142 — PR adds a general-purpose time-series API endpoint (GET /metrics/catalog/:kind/:namespace/:name/time-series) with supporting infrastructure (database query method, migration, validator middleware, utility functions, and new common types). While DORA metrics use history: true and defaultVisualization: 'sparkline' which benefit from this endpoint, the time-series API is architecturally distinct from the DORA plugin and serves all metrics. This could reasonably be a separate feature PR.
  Remediation: Consider whether the time-series API warrants a separate PR, or document in the PR description why bundling is intentional (e.g., the time-series endpoint is required for the DORA sparkline visualization to function).

Low

• [API contract] workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/GithubDeploymentWorkflowRunsCollector.ts — GithubDeploymentWorkflowRunsCollector output does not include the environment field in mapped deployment objects. The isProductionEnvironment function treats a missing/undefined environment as production (if (!environment) return true), so all workflow runs are treated as production deployments regardless of the productionEnvironments configuration. This is a design choice with clear rationale (workflow runs lack environment metadata from the GitHub API), but could surprise users who configure productionEnvironments expecting it to filter workflow-run-based deployments.
  Remediation: Document this limitation explicitly in the workflow runs collector documentation or in the app-config.yaml comments where the collector is referenced.

• [constructor-pattern] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.ts:264 — DoraDeploymentFrequencyProvider does not accept logger in its constructor, unlike its three sibling DORA providers (DoraChangeFailureRateProvider, DoraMeanTimeToRestoreProvider, DoraMedianLeadTimeForChangesProvider). Functionally correct since this provider does not currently log warnings for edge cases, but creates an inconsistent constructor shape within the module.
  Remediation: Consider adding logger to DoraDeploymentFrequencyProvider for consistency with sibling DORA providers, even if not currently used.

Previous run

Review

Findings

Low

• [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.ts:118 — When the collector returns deployments but none pass the isSuccessfulProductionDeployment filter, the provider returns 0 without distinguishing this from "no deployments at all". Other DORA providers (MedianLeadTime, ChangeFailureRate) throw an error when fewer than 2 successful deployments exist, giving the user feedback; this provider silently returns 0. Consider logging a warning when deployments exist but all are filtered out, to aid debugging misconfigured productionEnvironments.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts:101 — No guard against two adjacent successful production deployments sharing the same createdAt timestamp. Unlike DoraChangeFailureRateProvider which checks nextDeploymentCreatedAt <= deploymentCreatedAt and skips with a warning, this provider would pair two such deployments and attempt to resolve pull requests for baseCommitSha..headCommitSha where both SHAs may be identical. Add a check similar to DoraChangeFailureRateProvider (lines 129-139) that skips intervals where deployment.createdAt <= previousDeployment.createdAt and logs a warning.

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.test.ts — Missing test for the same-timestamp edge case. DoraChangeFailureRateProvider.test.ts includes a test for this scenario but DoraMedianLeadTimeForChangesProvider.test.ts does not.

• [injection-vuln] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/annotations/buildJqlFiltersFromEntity.ts:97 — Pre-existing: the customFilter annotation value (from jira/custom-filter) is passed through to JQL without sanitization, while other annotation values are validated via validateJQLValue(). This is carried forward from the old base.ts implementation. The new DORA incidents path (INCIDENT_FILTER_ANNOTATIONS) correctly omits customFilter. Consider documenting this as an intentional escape hatch.

• [scope-creep] workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts:142 — PR scope extends beyond the DORA plugin to include a time-series API endpoint, defaultVisualization field on the Metric type, CollectorConfig type in scorecard-common, database migration, GitHub module extensions, and Jira module refactoring. Consider updating the PR title/body to reflect the full scope.

• [scope-creep] workspaces/scorecard/plugins/scorecard/src/components/Common/CardWrapper.tsx — The tooltip wrapping change is a general UI improvement unrelated to DORA metrics.

• [naming-convention] workspaces/scorecard/plugins/scorecard-backend-module-github/src/github/GitHubClient.test.ts — Test file named GitHubClient.test.ts (capital H) doesn't match implementation file GithubClient.ts (lowercase h). This PR modifies the test file but does not fix the pre-existing casing mismatch. All new files correctly use the Github convention.

Previous run (2)

Review

Findings

Medium

• [stale-doc] workspaces/scorecard/plugins/scorecard-backend/README.md:97 — The "Available Metric Providers" table and the backend-module installation links omit the new DORA provider. Users consulting this central README to discover available providers will not find the DORA module.
  Remediation: Add a DORA row to the providers table and add the module installation link for @red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dora.

• [incorrect-doc] workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/deployment-frequency.md:53 — The collectors link uses ../scorecard-backend/docs/collectors.md which resolves to a non-existent path. The other three DORA metric docs correctly use ../../../scorecard-backend/docs/collectors.md (three levels up from docs/metrics/).
  Remediation: Change the link to ../../../scorecard-backend/docs/collectors.md.

Low

• [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts:169 — Negative lead times (where deployedAtTimestamp < firstCommitAtTimestamp due to clock skew) are silently skipped without diagnostic logging, making it difficult to debug calculation failures. Consider adding logger.warn when a PR is skipped due to negative lead time.

• [injection-vuln] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/annotations/buildJqlFiltersFromEntity.ts:79 — The customFilter annotation value (jira/custom-filter) is passed directly into JQL queries without validation, unlike project/component/label/team which use validateJQLValue. This is pre-existing behavior and customFilter is by design intended for arbitrary JQL from trusted entity annotation authors. Consider documenting this trust boundary explicitly.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/middlewares/validateTimeSeriesQueryParams.ts:25 — The time-series endpoint validates from <= to but does not enforce a maximum date range. A request spanning years could cause a large database scan. Consider adding a range cap (e.g., 365 days).

• [scope-creep] workspaces/scorecard/plugins/scorecard/src/components/Common/CardWrapper.tsx — The Tooltip wrapper change is a UI polish unrelated to DORA metrics. Consider splitting into a separate PR for cleaner history.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts:188 — Day-bucketing uses new Date(row.timestamp).toISOString().slice(0, 10) which is correct for Knex+pg (returns Date objects) but would break if the driver changed to return non-ISO strings.

• [scope-creep] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/annotations/annotationKeys.ts — The Jira annotation refactoring (enum → directory with multiple files) is substantial but functionally necessary for the DORA incident collector.

• [naming-convention] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/schemas/deploymentSchemas.ts — Schema naming diverges from the GitHub module (deploymentsCollectorOutputSchema vs deploymentsSchema), though justified by their different roles (consumer contract validation vs producer output).

Previous run (3)

Review

Findings

High

• [JQL Injection] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/annotations/buildJqlFiltersFromEntity.ts:93 — The customFilter annotation value (from entity annotation jira/custom-filter) is passed through to the JQL query without any sanitization or validation. When filterAnnotations.customFilter is set and the entity has a corresponding annotation, the raw annotation value is stored directly as filters.customFilter. This value flows into joinJqlClauses in openIssuesJql.ts via Object.values(entityFilters), where it is wrapped in parentheses and concatenated into the final JQL string. Unlike all other annotation fields (project, component, label, team) which go through sanitizeValue and validateJQLValue/validateIdentifier, the customFilter slot bypasses all sanitization. An attacker who can control entity annotations (via Backstage catalog YAML) could inject arbitrary JQL, potentially querying Jira projects they should not have access to. Note: only the open-issues path is affected — INCIDENT_FILTER_ANNOTATIONS does not include customFilter.
  Remediation: Either remove the customFilter annotation passthrough entirely, or apply validation to restrict its content (e.g., reject unbalanced parentheses, semicolons, and other JQL structural characters). A safer approach would be to parse the custom filter against an allowlist of JQL operators and field names.

Medium

• [scope-creep] The PR title says "introduce scorecard dora plugin" but the changeset is significantly broader than adding a new DORA plugin. It also: (1) refactors the entire Jira module by extracting JQL building into standalone functions, introducing an annotations subsystem, adding paginated request support to both Cloud and DataCenter strategies, and creating a JiraIncidentsCollector; (2) extends the scorecard-backend core with a new time-series API endpoint, database migration, and CatalogMetricService method; (3) extends scorecard-common with new types; (4) adds a CardWrapper tooltip change in the frontend. While these are coherent parts of the DORA feature, the PR touches 132 files across 6 packages — the description should explicitly call out the Jira module refactoring and backend core changes as distinct scope items.

Low

• [JQL Injection] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/clients/utils.ts:37 — The sanitizeValue function escapes backslashes and double quotes, then validateJQLValue checks the result against /^[a-zA-Z0-9 _-]+$/. The order of operations means sanitization is redundant: any input containing backslashes or quotes would be escaped into sequences that would then fail the strict allowlist regex. While the current behavior is safe (rejects bad input), the redundancy makes the security contract confusing.

• [JQL Injection] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/collectors/incidentJql.ts:42 — The issueType field in incidentsCollectorInputSchema uses z.string().min(1).optional() without a JQL-safe regex constraint. While validateJQLValue catches unsafe characters at JQL build time, validation happens late (at metric-calculation time rather than config validation time), making debugging harder.

• [config / schema mismatch] workspaces/scorecard/plugins/scorecard-backend-module-dora/config.d.ts:70 — config.d.ts declares thresholds?: ThresholdConfig for each DORA metric, but none of the parseDoraXxxConfig functions in DoraConfig.ts read or use this configuration — providers always use hardcoded defaults. The ThresholdResolver in scorecard-backend may handle this independently (consistent with other providers), but confirming end-to-end threshold override functionality would be valuable.

• [architecture-alignment] workspaces/scorecard/AGENTS.md — The Key files table was updated with the DORA metric ID reference, but was not updated to include the new DORA provider key files (DoraConfig.ts, DoraDeploymentFrequencyProvider.ts, etc.), despite listing equivalent files for all other providers.

• [architecture-alignment] workspaces/scorecard/examples/components/dora-scorecard.yaml — The example DORA scorecard component references a personal GitHub repository (dzemanov/test-scorecard-github-dora) rather than an organization-owned or clearly-test repository. Personal repos may become inaccessible if the developer leaves.

• [naming-convention] workspaces/scorecard/plugins/scorecard-backend-module-github/src/collectors/schemas/deploymentsSchemas.ts — Schema file naming inconsistency across packages: DORA module uses deploymentSchemas.ts (singular prefix) while GitHub collectors use deploymentsSchemas.ts (plural) and Jira uses incidentsSchemas.ts (plural).

• [constructor-visibility] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/collectors/JiraIncidentsCollector.ts:42 — JiraIncidentsCollector uses a public constructor while GitHub collector classes use private constructors with static fromConfig() factories. The difference has a practical reason (Jira shares one client across collectors) but the inconsistency across new collector classes is worth noting.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (4)

Review

Findings

Medium

• [error-handling-gap] workspaces/scorecard/plugins/scorecard-backend-module-github/src/github/GithubClient.ts:244 — In getCommitShasBetween, pagination has no fetchItemsLimit cap. Unlike getDeployments and getWorkflowRuns which use DEFAULT_DEPLOYMENT_FETCH_ITEMS_LIMIT, this method makes unbounded sequential API calls for large commit ranges, risking GitHub rate limits or timeouts.
  Remediation: Add a fetchItemsLimit parameter or reasonable upper bound on pages fetched, with a warning log when truncation occurs.

• [injection-vuln] workspaces/scorecard/plugins/scorecard-backend-module-jira/src/annotations/buildJqlFiltersFromEntity.ts:59 — The customFilter annotation value from catalog entity annotations is passed into JQL without sanitization. While other annotation-derived values (project, component, label, team) are protected by sanitizeValue/validateJQLValue/validateIdentifier, the customFilter slot is raw. This is a pre-existing design pattern that this PR extends to the new incident collector path via INCIDENT_FILTER_ANNOTATIONS.
  Remediation: Apply structural validation to customFilter values, or document that jira/custom-filter and jira/incident-custom-filter annotations allow arbitrary JQL and catalog write access should be restricted accordingly.

• [stale-doc] workspaces/scorecard/AGENTS.md:17 — The Metric ID Naming Convention section lists provider prefixes (github, jira, sonarqube, dependabot, openssf, filecheck) but does not include the new dora prefix introduced by this PR.
  Remediation: Add dora to the provider prefix list.

• [stale-doc] workspaces/scorecard/AGENTS.md:26 — The Complete Metric ID Reference section is missing the new DORA provider and its 4 metrics: dora.deploymentFrequency, dora.medianLeadTimeForChanges, dora.meanTimeToRestore, and dora.changeFailureRate.
  Remediation: Add a DORA section to the Complete Metric ID Reference table.

Low

• [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraChangeFailureRateProvider.ts:131 — The last deployment in the window is never evaluated as a failure source — incidents after the last deployment but within the 30-day window are silently ignored. This is a known DORA implementation trade-off requiring the next deployment to bound the interval.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.ts:115 — DeploymentFrequency returns 0 when no successful production deployments exist, while other DORA providers throw errors for insufficient data. This is semantically correct (0 frequency is a valid measurement) but differs from the error-throwing pattern used by the other three providers.

• [scope-creep] The PR scope extends beyond the title ("introduce scorecard dora plugin") to include a collectors framework with extension points, GitHub/Jira collectors, time-series API endpoint, database migration, new shared types, and significant refactoring of JiraClient and GithubClient. These form a cohesive dependency chain but documenting the broader scope in the PR description would improve reviewability.

• [naming-convention] workspaces/scorecard/plugins/scorecard-backend-module-github/src/github/GithubClient.ts — Pre-existing casing inconsistency between GithubClient.ts (implementation) and GitHubClient.test.ts (test file). All new code consistently uses Github (lowercase h).

• [pattern-inconsistency] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraConfig.ts — Uses import { JsonValue } instead of import type { JsonValue } when JsonValue is only used in type position.

• [new-package-dependency] workspaces/scorecard/plugins/scorecard-common/package.json — Adds @backstage/types (^1.2.2) as a new runtime dependency for CollectorConfig. This is near-zero-weight and already transitively present.

• [input-validation] workspaces/scorecard/plugins/scorecard-backend/src/middlewares/validateTimeSeriesQueryParams.ts:26 — No maximum time range enforced between from and to. The index and daily bucketing mitigate performance impact.

• [pattern-inconsistency] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/schemas/incidentSchemas.ts — Output schema omits .passthrough() unlike the parallel deployment output schema and all input schemas.

---

Labels: PR introduces a new DORA metrics plugin module with new API endpoints and collectors framework

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 8:16 AM UTC · Ended 8:26 AM UTC

Commit: 923d8ff · View workflow run →

[dzemanov]

For fullsend review: #4235 (comment)

• JQL Injection in customFilter: removed this option from incident filters and created https://redhat.atlassian.net/browse/RHDHBUGS-3646 to address it for jira.openIssues
• scope-creep: updated description
• sanitization is redundant: removed
• config / schema mismatch: scorecard backend handles thresholds. Thresholds are mentioned in all scorecard modules configurations, we can remove them + schedules from all of them in a separate PR if we would like to keep them only in scorecard-backend config.d.ts
• outdated AGENTS.md - fixed
• personal GH repo: will update it in following PR when I create repo in our test org as stated in feat(scorecard): introduce scorecard dora plugin #4235 (comment)
• naming-convention - fixed
• constructor-visibility - these classes are not exported

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 8:27 AM UTC · Ended 8:29 AM UTC

Commit: fd3800c · View workflow run →

[fullsend-ai-review]

🤖 Review · ❌ Terminated · Started 8:30 AM UTC · Ended 9:13 AM UTC

Commit: 2108e6e · View workflow run →

Superseded by updated review

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 8:30 AM UTC · Completed 9:13 AM UTC

Commit: 2108e6e · View workflow run →

[fullsend-ai-review]

🤖 Review · ❌ Terminated · Started 1:40 PM UTC · Ended 2:20 PM UTC

Commit: 62d5e86 · View workflow run →

[dzemanov]

Timeseries fixes in eef4df2:

• Added maximum range limit of 365 days for time-series API
• removed new Date(row.timestamp).toISOString().slice(0, 10) from day bucketing

Added warning logs for skipped invalid data for lead time for changes and change failure rate in eef4df2
Updated stale docs in 9c4726e

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 1:40 PM UTC · Completed 2:20 PM UTC

Commit: 62d5e86 · View workflow run →

[fullsend-ai-review]

🤖 Review · ❌ Terminated · Started 2:39 PM UTC · Ended 3:21 PM UTC

Commit: ad926c2 · View workflow run →

[sonarqubecloud]

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.3% Duplication on New Code

See analysis details on SonarQube Cloud

[dzemanov]

Fixed e2e in ad926c2 for metric group cards after merge of main, where multiple elements resolve:

[fullsend-ai-review]

See the review comment for full details.

---

Note: The following inline comments could not be posted on the diff (GitHub returned 422) and are included here instead:

• workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts:142: [medium] scope-creep

PR adds a general-purpose time-series API endpoint with supporting infrastructure (database query, migration, validator, utility functions, common types). While DORA metrics benefit from this, the time-series API is architecturally distinct and could be a separate feature PR.

Suggested fix: Consider whether the time-series API warrants a separate PR, or document why bundling is intentional.

• workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.ts (file-level): Line 264 · [low] constructor-pattern

DoraDeploymentFrequencyProvider doesn't take logger in its constructor, unlike its 3 sibling DORA providers. Functionally correct since it doesn't log warnings, but creates an inconsistent constructor shape.

Suggested fix: Consider adding logger for consistency with sibling DORA providers.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:39 PM UTC · Completed 3:21 PM UTC

Commit: ad926c2 · View workflow run →

[PatAKnight]

LGTM