fix(os-index): replace os:: prefix with .os suffix as DB uniqueness artifact

[fabrizzio-dotCMS]

Summary

Closes #35820

Why this change is safe

This code is still behind the OpenSearch feature flag and has never been rolled out to production. The OS index layer is entirely in active development, which means there is room for this kind of intentional renaming experiment to get the naming strategy right before any production data is involved.

.os suffix — why it matters for correctness

OpenSearch indices are registered under the same logical names as their ES counterparts (e.g. `working_20260528`). When APIs return a `Map<String, ...>` or a `Set` keyed by index name, ES and OS entries would silently collide — the OS entry would overwrite the ES one and data would be lost. Appending a `.os` suffix to every OS index name guarantees that both providers can coexist in the same map/set without any information loss.

Collision-free guarantee: logical index names always end in `_YYYYMMDDHHMMSS` (numeric timestamp) — they can never naturally end in `.os`.

Changes:

• IndexTag.OS now appends .os (suffix) instead of prepending os:: (prefix)
• isTagged() checks endsWith(".os"); untag() strips the last 3 chars
• IndexTag gains a suffix field alongside the existing prefix field — all existing callers unchanged
• VersionedIndicesAPIImpl comments updated to reflect the new storage format
• OPENSEARCH_MIGRATION.md — "tag ownership rule" section rewritten with the suffix rationale

No DB migration task: the OS index layer has never shipped to production, so there are no os::name rows to convert. Any dev/QA instance that ran an earlier build of this feature must drop its stale indicies OS rows (or let bootstrap recreate the OS indices) — there is no automatic conversion.

Improved bootstrap sync strategy

initOSCatchup() in ContentletIndexAPIImpl now follows a three-case priority order when OS indices are absent from the cluster:

1. OS DB record exists, cluster index missing — recreate the OS index using the name already registered in VersionedIndicesAPI. This is the authoritative source in Phase 3.
2. OS DB empty, ES DB has a record (migration catch-up, Phases 1/2) — mirror the ES logical name so both indices share the same base name. This prevents timestamp divergence — e.g. the ES index working_20240101 produces working_20240101.os, not a freshly-stamped working_20260528.os.
3. Both empty (fresh install or data-loss) — generate a new timestamp and emit a warning for the operator.

Tests — OSIndexAPIImplIntegrationTest

Two stub tests that only asserted "returns non-null" were upgraded to real integration assertions:

• test_getClusterHealth_shouldReturnHealthForCreatedIndex — creates a live index, then verifies the health map contains an entry for it with a non-null/non-empty status and positive shard count.
• test_getIndicesStats_shouldReturnStatsForCreatedIndices — creates live and working indices, then verifies both appear in the stats map with non-negative document count, non-negative raw size, and a non-empty human-readable size string.

Three new phase-routing tests were also added to ContentletIndexAPIImplMigrationIT to verify that getIndicesStats and getClusterHealth correctly merge ES + OS results in Phase 1 and return ES-only results in Phase 0.

Test plan

• Verify IndexTag.OS.tag("cluster_abc.working_20260528") returns "cluster_abc.working_20260528.os"
• Verify IndexTag.strip("cluster_abc.working_20260528.os") returns "cluster_abc.working_20260528"
• Verify IndexTag.resolve("cluster_abc.working_20260528.os") == IndexTag.OS
• Verify IndexTag.resolve("cluster_abc.working_20260528") == IndexTag.ES (untagged default)
• On a fresh instance: confirm Phase 1 bootstrap stores name.os in DB and portlet shows correct indices
• Confirm getIndicesStats / getClusterHealth maps in Phase 1 contain both ES and OS entries (no collision/overwrite)
• No regression on ES portlet behavior
• OSIndexAPIImplIntegrationTest — test_getClusterHealth_shouldReturnHealthForCreatedIndex and test_getIndicesStats_shouldReturnStatsForCreatedIndices pass against a live OS cluster

🤖 Generated with Claude Code

[claude]

Pull Request Unsafe to Rollback!!!

• Category: H-1 — One-Way Data Transformation (Storage Format Change)
• Risk Level: 🟠 HIGH
• Why it's unsafe: The entire diff (from the previous analysis baseline through commit 8a8ada82) establishes .os as the end-to-end canonical OS index name — not just a DB artifact. Previously, VersionedIndicesAPIImpl.loadIndices() stripped the tag before returning names to callers (so clean names cluster_x.working_ts were exposed). After 8a8ada82, the private stripTags() helper is deleted and loadIndices() / loadAllIndices() / loadNonVersionedIndices() return the raw .os-suffixed form (cluster_x.working_ts.os) to every caller. If rolled back to N-1 after OS was active, N-1's loadIndices() would call the now-deleted stripTags() which expected to strip os:: (the old prefix form) — it no longer exists. N-1 callers receive cluster_x.working_ts.os as a logical name, which N-1's routing treats as an untagged ES name (IndexTag.resolve("name.os") == ES on N-1 since startsWith("os::") is false) and routes OS operations to the ES provider — data reads silently return wrong results.
• Code that makes it unsafe:
  • dotCMS/src/main/java/com/dotcms/content/index/VersionedIndicesAPIImpl.java — private static VersionedIndices stripTags(VersionedIndices) deleted; loadIndices(), loadAllIndices(), loadNonVersionedIndices() no longer transform the DB-loaded .os-suffixed names — callers now receive the tagged form end-to-end
  • dotCMS/src/main/java/com/dotcms/content/index/IndexTag.java — OS("", ".os"): isTagged uses endsWith(".os"); N-1 isTagged uses startsWith("os::") → false for .os-suffixed names — routing silently defaults to ES
• Mitigating factors:
  • The OS index layer is behind FEATURE_FLAG_OPEN_SEARCH_PHASE; N-1 with the flag disabled will never read OS rows
  • These names were never written in production — risk is limited to dev/staging environments where the flag has been enabled
• Alternative: Before rolling back in any environment where the OS feature was active, disable FEATURE_FLAG_OPEN_SEARCH_PHASE first so N-1 never reads the OS rows; or manually revert indicies rows from name.os back to os::name format.

---

• Category: M-4 — OSGi Public Interface Change (RESOLVED by commit 8a8ada82)
• Risk Level: ✅ RESOLVED
• Previously: IndexAPI.removeClusterIdFromName() was abstract in an earlier commit — any OSGi plugin relying on the former default impl would have failed with AbstractMethodError.
• Now: Commit 8a8ada82 converts removeClusterIdFromName, getNameWithClusterIDPrefix, getClusterPrefix, and hasClusterPrefix to default methods on IndexAPI — backward-compatible. Existing plugin implementations continue to work unchanged. The M-4 risk from the prior analysis no longer applies.

[claude]

Pull Request Unsafe to Rollback!!!

• Category: H-1 — One-Way Data Transformation (Storage Format Change)
• Risk Level: 🟠 HIGH
• Why it's unsafe: The commit range analyzed (c036699 → 221088f) removes VersionedIndicesAPIImpl.stripTags() and changes loadIndices(), loadAllIndices(), and loadNonVersionedIndices() to return the raw .os-suffixed canonical form to every caller (previously those methods stripped the tag before returning). On N-1, IndexTag.OS.isTagged() checks startsWith("os::") — it returns false for the new .os-suffixed names. As a result, any caller that received cluster_x.working_ts.os from the DB after rollback to N-1 would have IndexTag.resolve() classify it as ES (the unrecognized-tag default), silently routing OS reads/writes to the ES provider and returning wrong results.
• Code that makes it unsafe:
  • dotCMS/src/main/java/com/dotcms/content/index/VersionedIndicesAPIImpl.java — private static VersionedIndices stripTags(VersionedIndices) deleted; loadIndices(), loadAllIndices(), loadNonVersionedIndices() now return DB names unchanged (cluster_xxx.working_ts.os) instead of stripping the tag to cluster_xxx.working_ts as N-1 expects
  • dotCMS/src/main/java/com/dotcms/content/index/IndexTag.java — OS("os::", "") changed to OS("", ".os"): N-1 isTagged checks startsWith("os::") → false for .os-suffixed names → routing silently defaults to ES
• Mitigating factors:
  • The OS index layer is behind FEATURE_FLAG_OPEN_SEARCH_PHASE; N-1 with the flag disabled never reads OS rows — risk is limited to dev/staging environments where the flag has been enabled
  • These names were never written in production (feature not yet shipped)
• Alternative: Before rolling back in any environment where the OS feature was active, disable FEATURE_FLAG_OPEN_SEARCH_PHASE first so N-1 never reads the OS rows; or manually revert indicies rows from name.os back to os::name format before downgrading.

---

• Category: M-4 — OSGi Public Interface Change
• Risk Level: ✅ RESOLVED
• Previously: IndexAPI.removeClusterIdFromName() was abstract in an earlier commit — any OSGi plugin relying on the former default impl would have failed with AbstractMethodError.
• Now (commit 8a8ada82): removeClusterIdFromName, getNameWithClusterIDPrefix, getClusterPrefix, and hasClusterPrefix are all default methods on IndexAPI; ESIndexAPI and OSIndexAPIImpl dropped their duplicate impls and inherit the defaults. Existing OSGi plugin implementations continue to work unchanged. The M-4 risk no longer applies.