[Cosmos] Backport PPCB fixes to 4.79.1-hotfix - #50281
Draft
Abhijeet Mohanty (jeet1995) wants to merge 14 commits into
Draft
[Cosmos] Backport PPCB fixes to 4.79.1-hotfix#50281Abhijeet Mohanty (jeet1995) wants to merge 14 commits into
Abhijeet Mohanty (jeet1995) wants to merge 14 commits into
Conversation
…ientCfgs The partitionLevelCircuitBreakerCfg field disappeared from the clientCfgs section of CosmosDiagnostics when a customer explicitly enabled Per-Partition Circuit Breaker (PPCB) client-side. Root cause: the diagnostics write was coupled to the Per-Partition Automatic Failover (PPAF) initialization path, so the field was only populated when the service mandated PPAF, not when PPCB was configured client-side. Fix: move the diagnostics write into initializePerPartitionCircuitBreaker(), which is invoked unconditionally at client init, so the field appears whenever the circuit breaker is configured client-side. Adds a CI-runnable regression test asserting all clientCfgs keys are present, including partitionLevelCircuitBreakerCfg. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…roperty Remove trailing comma before closing brace in the COSMOS.PARTITION_LEVEL_CIRCUIT_BREAKER_CONFIG JSON so the value is strictly valid and clearer as a customer reference. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…kerE2ETests Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… keys The partitionLevelCircuitBreakerCfg field now appears in clientCfgs for every client regardless of PPCB configuration, so the E2E test no longer sets the COSMOS.PARTITION_LEVEL_CIRCUIT_BREAKER_CONFIG system property. It just builds a plain client and asserts all expected clientCfgs keys (including partitionLevelCircuitBreakerCfg) are present in CosmosDiagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove PPCB System.setProperty/clearProperty and the value-specific assertion; assert only that all clientCfgs keys are present, matching the E2E test simplification. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Fix PPCB failback with missing or stale addresses --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve PPCB failback diagnostics * Validate PPCB state in diagnostics E2E tests * Limit PPCB diagnostics assertion to data-plane requests * Log PPCB failback backlog progress * Add PPCB failback remaining meter * Clarify PPCB failback meter name * Reduce PPCB failback meter allocations * Track PPCB pending recoveries by collection * Correlate PPCB failback recovery diagnostics * Harden PPCB failback recovery tests * Fix PPCB all-region diagnostics assertion * Optimize PPCB diagnostics snapshots * Cache PPCB diagnostics snapshots * Align PPCB failback flow with main * Add benchmark fault injection support. * Harden PPCB failback telemetry Use a single injectable logger, prevent backlog telemetry failures from escaping the recovery flow, and align the pending failback metric name. * Scope PR to PPCB diagnostics Remove benchmark, fault-injection, metric, and recovery behavior changes. Retain immutable CosmosDiagnostics PPCB snapshots, lifecycle E2E assertions, and WARN logging for every failback failure. * Reduce PPCB diagnostics overhead Reuse immutable PPCB map references in response snapshots and shorten per-region diagnostic field names. * Refactoring * Simplify PPCB diagnostics snapshots Represent holder state with one volatile immutable-map reference and align the compact timestamp serialization test. * Ignore updates to empty PPCB diagnostics Make updates to the shared uninitialized diagnostics sentinel a no-op and verify it remains null-serializing. * Avoid copying PPCB diagnostics state Retain the live PPCB diagnostics map reference to avoid per-publication map and wrapper allocations, accepting weak consistency. * Log PPCB diagnostics lifecycle snapshots Emit one full CosmosDiagnostics JSON payload for failed, post-failover, and post-failback E2E phases for PR evidence. * Document PPCB diagnostics improvements Add the unreleased changelog entry for per-region PPCB snapshots and failback WARN logging. * Add PPCB failback outcome diagnostics Track the latest background failback attempt time, outcome, and failure reason per partition-region; validate lifecycle state in E2E and focused recovery tests. * Reset unavailable timestamp after PPCB failback Use the available-state sentinel when recovery moves a region to HealthyTentative and cover it in the scheduled recovery test. * Refine PPCB failback diagnostics Keep failback attempt metadata partition-scoped, retain only the latest full failure message per region, and clear retained messages when recovery backlog drains.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 16ce0941-555c-4190-8c4d-96c705086350
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 16ce0941-555c-4190-8c4d-96c705086350
|
Azure Pipelines: 34 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 16ce0941-555c-4190-8c4d-96c705086350
Complete the existing pipeline authentication flow with mirror settings so network-isolated agents resolve Maven dependencies through the Azure SDK feed.
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backports the required PPCB fixes to
azure-cosmos4.79.1-hotfix.Comparison baseline
ppcb-hotfix-baseline/azure-cosmos-4.79.0896d7f94f1ea82d8da16f26fa69100a6e82f17f9jeet1995:release_azure-cosmos-4.79.1-hotfixBackport inventory
d70208a69af00b8961feb2f4f6ac73ffb54c,310ed771c401,572af904642e,a440ab76c29a,a796cc58d04c,f3f4b708437d,76f73b7fd78ac2598b9e7517Release integration commits
7cfe6ec04436- prepare 4.79.1-hotfix versions and release metadata.53a49a71915c- adapt PPCB tests to the 4.79 release surface.a49d8e5c09f0- restore PPCB regression coverage removed during release adaptation.dd091bc2ddbc- complete authenticated Azure Artifacts Maven resolution for network-isolated pipelines.Validation
azure-cosmos,azure-cosmos-test, andazure-cosmos-testsbuild successfully in the local four-hotfix harness.