Repository navigation
feat(kms)!: refuse to serve without a key handover path - #1564
Merged
kvinwang merged 3 commits intoNov 11, 2026
Merged
Conversation
With the public handover off by default, other KMS nodes can only onboard through the admin listener over TLS. A KMS without either would strand them, which shows up only when a node needs to join. Before serving, the KMS now requires core.onboard.public_key_handover = true, or the admin listener enabled with [core.admin.tls] key and certs, and names the missing setting otherwise. There is no opt-out, dev auth included. The onboarding phase is not checked. The built-in kms.toml now enables the admin listener with TLS on the RPC key and certificate. The admin listener fails closed, so a KMS on the default config needs an admin token to start. A KMS deployed as a CVM with the local key provider has no secret channel for its config: the app compose is measured and public. The new core.admin.auth_token_hash takes the token's hex SHA-256, so the compose carries only the hash: - dstack-app composes and entrypoint use the ADMIN_TOKEN_HASH the deploy scripts already compute, and the scripts publish the admin listener when KMS_ADMIN_ADDR is set. - dstackup generates a KMS admin token next to the VMM one and renders its hash. - tools/dev-stack.sh generates a local token. - The full-stack e2e KMS, shared with KMS 0.5.8, keeps the admin listener off and serves the public handover. Signed-off-by: Kevin Wang <wy721@qq.com>
Checked only before serving, a misconfigured node completed onboarding or bootstrap, getting or generating root keys, and only then failed. Check at startup instead, in every mode. Also document that an auth_token_hash, public in an app compose, needs a high-entropy random token: a weak one can be brute-forced offline from its hash. Signed-off-by: Kevin Wang <wy721@qq.com>
With the default config and no admin token, the handover check passed (admin enabled, TLS paths set) but the missing credential surfaced only when the admin listener was built, after onboarding or bootstrap. Build the admin listener setup right after the handover check, so such a node also fails before it gets or generates root keys. Signed-off-by: Kevin Wang <wy721@qq.com>
kvinwang
added a commit
that referenced
this pull request
Nov 11, 2026
Since #1564 a KMS refuses to start without a key handover path, and the default config enables the admin listener with TLS at /etc/kms/certs. - The isolated KMS fixtures serve admin TLS with their own RPC key and certificate, so their admin URLs are https. - tc-kms-onboard-005 no longer adds admin TLS before disabling the public handover; the fixture already has it. - tc-kms-startup-001 calls the target's admin listener over https. - Drop the public_key_handover line this branch added to the full-stack KMS config; #1564 added the same key there. Signed-off-by: Kevin Wang <wy721@qq.com>
kvinwang
added a commit
that referenced
this pull request
Nov 11, 2026
Since #1564 a KMS refuses to start without a key handover path, and the default config enables the admin listener with TLS at /etc/kms/certs. - The isolated KMS fixtures serve admin TLS with their own RPC key and certificate, so their admin URLs are https. - tc-kms-onboard-005 no longer adds admin TLS before disabling the public handover; the fixture already has it. - tc-kms-startup-001 calls the target's admin listener over https. - Drop the public_key_handover line this branch added to the full-stack KMS config; #1564 added the same key there. Signed-off-by: Kevin Wang <wy721@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With the public handover off by default (#1552), other KMS nodes can only onboard through the admin listener over TLS. A KMS that has neither leaves new nodes unable to join, and nobody notices until a node actually tries.
core.onboard.public_key_handover = trueor the admin listener with[core.admin.tls]keyandcerts. If neither is configured, it refuses to start and the error names the missing setting and both ways to fix it. There is no opt-out,auth_api = "dev"included.kms.toml: the admin listener and its TLS are now on, using the RPC key and certificate. The admin listener refuses to start without a credential, so a KMS on the default config needs a token.core.admin.auth_token_hash: a KMS running in a CVM with the local key provider can't receive secrets, and its app compose is public. This setting takes the token's hex SHA-256, so the compose carries only the hash. The hash is public there, so the token must be high-entropy random (openssl rand -hex 32).dstack-appcomposes and entrypoint use theADMIN_TOKEN_HASHthat the deploy scripts already compute. The scripts publish the admin listener whenKMS_ADMIN_ADDRis set.dstackupgenerates a KMS admin token and renders its hash into the config.tools/dev-stack.shgenerates a local token.deployment.md,upgrading-to-0.6.md, both KMS tutorials, the rc runbook and CHANGELOG.Tests:
auth_token_hash. dstack-cli-core 31 and dstackup 17 also passed.Real binary runs (debug
dstack-kms, dev auth,attest_rpc_cert = false, onboard enabled, fresh cert dir each time):auto_bootstrap_domain)[core.admin.tls] key = ""[core.admin.tls] keyauth_token_hashpublic_key_handover = trueOn the serving node, admin over HTTPS with the plaintext token behind
auth_token_hashreturns 200; a wrong or missing token returns 401.