docs: add cluster prerequisites and troubleshooting for node join - #247
docs: add cluster prerequisites and troubleshooting for node join#247nnamuhcs wants to merge 1 commit into
Conversation
The node steps assume an existing AKS cluster is already prepared with the Flex Node cluster-side components (CNI, aks-flex-controller with the CSR approver enabled, and a registered machine per node). On a plain cluster that is not the case, and 'aks-flex-node start' fails with 'wait for daemon controller certificate: context deadline exceeded'. - Add a Prepare the cluster section listing the three cluster-side requirements, with the controller deploy command and pointers to the lab guides, controller-deployment assets, and design docs. - Add a Troubleshooting section mapping the certificate-timeout symptom and Pending CSRs to the missing or misconfigured controller. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a3bcbde4-d79a-489e-98fa-fcc66643363e
|
Thanks for the contribution! This pull request comes from a fork, so the Azure E2E workflow is intentionally skipped for security reasons. Merge will remain blocked until the E2E tests have been run from a branch in the Maintainer options:
We do not run Azure E2E directly from fork PR code because it requires Azure OIDC access. |
There was a problem hiding this comment.
Pull request overview
This PR updates the project’s primary quickstart documentation to include required cluster-side prerequisites for successfully joining Flex Nodes, and adds a focused Troubleshooting section for the most common node-join failure mode (daemon certificate approval timeouts).
Changes:
- Adds a new Prepare the cluster section documenting cluster-side requirements (CNI,
aks-flex-controllerdeployment, machine registration). - Adds a Troubleshooting section mapping
wait for daemon controller certificate: context deadline exceededto missing/disabled controller CSR approval or missing machine registration.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 2. **The AKS Flex Controller** in `kube-system`, with the CSR approver enabled. It approves each node's daemon certificate and serves machine goal state. Deploy it from [`hack/controller-deployment/`](hack/controller-deployment/), ensuring the deployment runs with `--enable-csr-approver=true` (the checked-in manifest defaults to `false`): | ||
|
|
||
| ```bash | ||
| kubectl apply -k hack/controller-deployment/ | ||
| kubectl -n kube-system rollout status deployment/aks-flex-controller | ||
| ``` |
| # Is this node registered as a machine? | ||
| kubectl -n kube-system get configmap aks-flex-machines -o json | jq -r '.data | keys[]?' |
|
|
||
| ### Prepare the cluster | ||
|
|
||
| Before you join any nodes, the target AKS cluster must be set up with the Flex Node cluster-side components. A plain AKS cluster does **not** have them, and `aks-flex-node start` (Step 6) will hang, then fail with `wait for daemon controller certificate: context deadline exceeded` (see [Troubleshooting](#troubleshooting)). |
There was a problem hiding this comment.
I don't hanging the process is true? We could say the lifecycle daemon controller will be in crashing state, but this is not blocking the kubelet bootstrapping
There was a problem hiding this comment.
and user can get it working by manually approving the CSR generated by the lifecycle daemon controller
| The cluster needs three things: | ||
|
|
||
| 1. **A Flex-compatible CNI.** A default AKS CNI does not route pod traffic to Flex nodes. The validated CNI is [Unbounded-Net](https://github.com/Azure/unbounded); the [lab guides](docs/labs/README.md) create the cluster with `--network-plugin none` and install it. | ||
| 2. **The AKS Flex Controller** in `kube-system`, with the CSR approver enabled. It approves each node's daemon certificate and serves machine goal state. Deploy it from [`hack/controller-deployment/`](hack/controller-deployment/), ensuring the deployment runs with `--enable-csr-approver=true` (the checked-in manifest defaults to `false`): |
There was a problem hiding this comment.
we should remove mentioning for this aks-flex-controller because it's not expected to deploy on user's own
Summary
Follow-up to #244. That PR rewrote Getting Started into a guided quickstart for the node side. This PR adds the cluster-side setup that the node steps depend on, plus a Troubleshooting section.
Today the docs walk a user through
generate-node-config→install→start, but they don't mention the cluster-side components the agent needs. Against a plain AKS cluster,aks-flex-node starthangs and then fails with:The cause is that nothing approves the node's daemon certificate: the in-cluster
aks-flex-controller(CSR approver) isn't deployed, or the machine isn't registered. The daemon CSRs (kubernetes.io/kube-apiserver-client) sitPendingand the agent times out.What this PR adds
aks-flex-controllerdeployed inkube-systemwith--enable-csr-approver=true(the checked-in manifest defaults tofalse), including thekubectl apply -k hack/controller-deployment/command;kube-system/aks-flex-machinesConfigMap, pointing todocs/design/in-cluster-machine.mdandhack/e2e/.kubectlchecks.Documentation only; no code or command changes. Since the cluster-side setup is currently exercised mainly through
hack/e2e, I'd welcome a maintainer sanity-check on the Prepare the cluster steps, and any preference on whether this belongs in the README quickstart or the lab guides.