Repository navigation
Conversation
kvinwang
force-pushed
the
feat/kms-epochs-combined
branch
4 times, most recently
from
November 8, 2026 02:57
51f8598 to
933ffb6
Compare
This was referenced Nov 8, 2026
kvinwang
force-pushed
the
feat/kms-epochs-combined
branch
2 times, most recently
from
November 8, 2026 03:34
4ed6d0d to
1de3c9e
Compare
kvinwang
force-pushed
the
feat/kms-epochs-combined
branch
from
November 8, 2026 07:26
1de3c9e to
96edd77
Compare
Describe recovery from KMS root-key exposure using a fleet-wide maintenance window: per-epoch roots registered on DstackKms, a single GetAppKey epoch selector (absent = epoch 0, -1 = latest), LUKS2 keyslot rotation with an epoch token, in-CVM env re-encryption, full-history KMS onboarding with a rotation mode, and a clean gateway reset. Signed-off-by: Kevin Wang <wy721@qq.com>
DstackKms keeps an append-only list of root epochs (CA and k256 public keys, the URL of the attestation proving their generation, and a keccak256 chain hash over all epochs so far) and a separately activated current epoch. Registration and activation are distinct steps so every KMS instance can import an epoch before it goes live. A pending epoch can be popped when the nodes holding it are lost before it is activated. Epoch 0 must be the kmsInfo keys when those are set. The attestation stays off chain: its report data binds the keys, so it needs no on-chain hash, and storing it would cost about 720 gas per byte. The KMS sends the root epoch it serves and its chain hash as rootEpoch/rootEpochHash in the boot info; every auth backend (auth-eth, auth-eth-bun, auth-simple, auth-mock, dstack-auth) denies a KMS that does not serve the current epoch with the registered keys. A KMS that predates root epochs reports none and is cut off once a later epoch is current. Info responses report the current epoch so a KMS can pick it at startup. dstack-types gains the shared chain hash computation and test vectors. Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Evidence updates are recoverable and mismatched evidence fails verification. Revert the unnecessary chain hash requirement for URL updates; retain it for activation and removal. Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
kvinwang
force-pushed
the
feat/kms-epochs-combined
branch
3 times, most recently
from
November 8, 2026 14:14
1a7dd23 to
a5457bf
Compare
Signed-off-by: Kevin Wang <wy721@qq.com>
The KMS stores every root epoch it holds (the newest at the existing paths, older ones under epochs/<n>/) and at startup serves the epoch the auth backend reports as current, provided its own keys produce the registered chain hash. Every boot check carries the served epoch and its chain hash so the backend can deny a stale or forked KMS; on a denial the KMS re-reads the current epoch and exits to restart onto it if it moved. GetAppKey takes an optional root_epoch (unset = current) so guests can fetch the keys of an older epoch to migrate, and responses carry the epoch served. Onboarding hands over the whole keyring and checks it against the chain. With rotate, the new KMS also generates the next epoch and attests it; the source must then be at the current epoch and pass the strict KMS authorization check. Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
fix(auth-eth): refuse legacy authorization once a later root epoch is current
Prove app-epoch monotonicity, the epoch bound on policy writes, rejection of inactive epochs and foreign operations, legacy isAppAllowed equivalence at the current epoch, and epoch isolation of authorization decisions. Mutation testing confirms each property catches a targeted bug. Document the FOUNDRY_DYNAMIC_TEST_LINKING=false workaround needed to run Halmos 0.3.3 with Forge 1.8. Signed-off-by: Kevin Wang <wy721@qq.com>
Add positive properties: authorizeApp allows exactly when the epoch policy admits the app, and each per-epoch setter takes effect. Signed-off-by: Kevin Wang <wy721@qq.com>
Prove the root registry step invariant (append-only entries, chain hash recursion, hash-bound activation and pop, kmsInfo following the current epoch), canonical key encodings, authorization epoch bounds and root epoch reporting, bootstrap-only authorization before registration, KMS and app authorization matching epoch policy, epoch isolation of KMS policy writes, and legacy entry points following the current epochs. Mutation testing confirms each property catches a targeted bug. Signed-off-by: Kevin Wang <wy721@qq.com>
…acts Follow the interface of #1542: AuthorizationRequest and AuthorizationResult live in IAppEpochAuth.sol, `isAllowed` is `authorized`, and DstackApp's authorizeApp takes the request. Update the properties that the review intentionally changed: - *AtEpoch setters may prepare the next app epoch, and revert past it; - the legacy isAppAllowed refuses at a nonzero app epoch, and at epoch 0 decides at epoch 0 even with the next row prepared; - epoch-less KMS setters write every row from the current epoch to the latest one; - the legacy isKmsAllowed decides at root epoch 0. Each changed property was mutation-tested against the matching bug. Signed-off-by: Kevin Wang <wy721@qq.com>
Model the root epoch registry and KMS onboarding, data disk key migration with crashes, and the VMM env update window, as implemented, and check them with TLC. The models find that a crash during luksHeaderRestore can destroy the volume key, that an initialization interrupted before the epoch token is written fails every later boot, that lost boot notifications leave the env update window open to application code, and that an ordinary join can hand over a losing pending root. A model of a fix for the two disk issues passes. Signed-off-by: Kevin Wang <wy721@qq.com>
…tation DiskMigration now models the two-keyslot migration of #1539 (luksAddKey, token bound to its keyslot, luksKillSlot) instead of luksHeaderRestore, and lets the KMS rotate at any point of a boot. All invariants hold for epochs 0-3; the torn-restore script and the configs for the replaced scheme go. KmsRotation checks the latest registered epoch's chain hash on handover, as onboard_service.rs does since #1542, so HandoverAgreesWithRegistry now holds and joins the main config. EnvUpdate gains FailClosed for the guest of #1546, which holds with a lossy host API; the lossy config keeps the earlier guest as a counterexample. The README records the mutations each model catches. Signed-off-by: Kevin Wang <wy721@qq.com>
Since #1545, isKmsAllowed and isAppAllowed refuse while a later root epoch is current. Replace the epoch-0 equivalence check, which setUp's rotated state no longer reaches, with two checks: both entry points refuse after a rotation, and before any rotation isKmsAllowed decides as kms_serve at epoch 0 even with the next epoch registered and prepared. Removing either rotation guard or reading the pending row each produce a counterexample. Signed-off-by: Kevin Wang <wy721@qq.com>
test(auth-eth): add Halmos symbolic tests for app and root epochs
docs(formal): add TLA+ models of the key rotation protocol
Only auth-eth-bun (on-chain) and dstack-auth (file-based) remain as maintained authorization backends; every protocol change had to be repeated in each one. Nothing in CI or the docs ran auth-mock. For development, the KMS's built-in `type = "dev"` auth API allows everything. Signed-off-by: Kevin Wang <wy721@qq.com>
chore(kms): remove the auth-mock webhook backend
…B checks Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
…th-dstack-auth chore(kms): replace auth-simple with dstack-auth
auth-eth-bun serves the same webhook API from the same contracts, so the Fastify/ethers server in dstack/kms/auth-eth is a second implementation to keep in sync. Remove it together with its Jest tests, npm manifests and the integration scripts that only existed to run it. dstack/kms/auth-eth keeps everything the contracts need: contracts/, Foundry tests, script/*.s.sol, lib/ submodules, foundry.toml, Slither config, .openzeppelin/ and the formal-verification docs. The README now covers only the contracts, plus a short Anvil + auth-eth-bun recipe that replaces scripts/setup-local-chain.sh. kms/dstack-app/compose-dev.yaml builds auth-eth-bun from GIT_REPOSITORY at GIT_REV instead of the Node server, and deploy-to-vmm.sh passes through ETH_CHAIN_ID and ETH_FINALITY_CONFIRMATIONS. Docs and tutorials that ran the Node server now run auth-eth-bun; the contract tutorial uses forge. Signed-off-by: Kevin Wang <wy721@qq.com>
Signed-off-by: Kevin Wang <wy721@qq.com>
chore(kms): remove the Node auth-eth webhook server
chore(kms): rename auth-eth-bun to auth-eth
Collaborator
Author
|
@Leechael Thanks for the thorough review. Below is how each item in the review (against The follow-up PRs from this review (#1539, #1540, #1542, #1543, #1544) are all merged into Blocking
Should-fix
Crash consistency and resource exhaustion
Changes that came out of the discussion
Not yet decided
Still pending: integration tests in #841 (not merged yet) for Should-fix 4, Should-fix 5, and the end-to-end part of Test coverage. |
The v0.5.9 and v0.6.0 sources under contracts/test-utils/reference are kept verbatim as upgrade references, so trailing-whitespace and end-of-file-fixer must not rewrite them. Signed-off-by: Kevin Wang <wy721@qq.com>
…ontracts ci(prek): skip whitespace fixers on reference contracts
Root-key handover now lives on the admin listener, so the public KMS.GetKmsKey entry is only needed while a 0.5.x KMS, or an onboarding request without source_token, still onboards from the cluster. Default core.onboard.public_key_handover to false in kms.toml and in the serde default, and make the refusal name the option to enable. The onboarding test runbooks onboard without a source token, so they now set the option on the source KMS explicitly. Signed-off-by: Kevin Wang <wy721@qq.com>
…r-off feat(kms)!: turn off the public KMS key handover by default
This was referenced Nov 11, 2026
kvinwang
added a commit
that referenced
this pull request
Nov 11, 2026
Since #1533, KMS key release verifies a TDX caller's OS image. The SDK simulator replays a captured legacy quote whose image (e61be43d...) was never published, so every simulated KMS client failed at the image download. Derive the KMS fixtures' simulator files with the candidate dstack-tee-simulator instead: it computes the quote measurements from the lite tdx_measurement document of the verifier fixture, the same way the verifier does, and the captured RTMR3 events are replayed on top. The verifier then takes the lite path and downloads nothing. The simulator runs in a private user and mount namespace because the TDX ABI paths under /sys are fixed. Signed-off-by: Kevin Wang <wy721@qq.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Register and rotate root/app key epochs, retain authorized historical keys, migrate disk headers and SealedEnv, and enforce epoch-scoped authorization while preserving temporary-CA compatibility.
This aggregates the linear split stack:
#1516 → #1519 → #1517 → #1520 → #1521 → #1528 → #1530 → #1532.
Restores all original feature commits after the previous stack rewrite omitted application epoch support. Includes the review fixes from #1534: optional selectors, grouped epoch fields, scoped operation names, consistent authorization methods, removal of legacy KMS setters, consolidated app epoch policy interface, and RPC injection/isolation regression coverage. No KMS contract-address equality check was added.
The final tree is identical to #1534 at
2c017875bf. It was independently compared against the original complete #1533/#1534 tree plus the intended review fixes; no unrelated feature changes remain.Validation on Linux x86_64 with Rust 1.92.0, Bun 1.4.4, Forge 1.8.4 and Solc 0.8.24: cert-client 7, KMS 75, local auth 9, Solidity 87, VMM SealedEnv 4, simple auth 33, mock auth 15, Bun Ethereum auth 50 and Jest Ethereum auth 22 tests passed. Guest-agent and dstack-util compile checks passed, including the earlier guest-key split independently. Solidity used pinned submodules and
forge test --offline --ffi -vv; Rust reused Cargo caches. Injection tests use loopback HTTP, a supplied vsock CID, synthetic payloads and temporary VM storage; no live TEE/chain or kernel AF_VSOCK identity test was performed. Tree equality, stack ancestry, Rust formatting and diff whitespace checks passed.GitHub currently locks the base of #1519 because it is a stack member. Its commits descend from #1516, but its PR base remains
next; both GraphQL and REST base updates were rejected.