Skip to content

Commit fb88138

Browse files
snarkychefsnarkychef
authored
Disable kubelet fail on cgroupv1 on kind (#5143)
1 parent ef3b413 commit fb88138

9 files changed

Lines changed: 75 additions & 18 deletions

‎projects/kubernetes-sigs/kind/CHECKSUMS‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
6efda113dea98399fbc624d9a415f3fabf9281231095adf17cd135495c651842 _output/bin/kind/linux-amd64/kind
2-
44b5eae4023903016ba5b6872513cfd8b804d861f74d903b05669c691c405ff6 _output/bin/kind/linux-amd64/kindnetd
3-
73a0ef28be2fe7609d8ebb283c48f14e5324eaf79b237927ffb7e5a6547d5ec3 _output/bin/kind/linux-arm64/kind
4-
4729ab4410e722685499d12812a3306070a1a1b1e2a3a2e6628366128200b940 _output/bin/kind/linux-arm64/kindnetd
1+
7f22aca498cfe51e752d738c82cbab251868aefc4a086081f2ad3f433ad6274b _output/bin/kind/linux-amd64/kind
2+
4bf9ed9ddc1b4bc22d53a1a8119de2e6e05e13924f4ee78541f695d4f58d7d0d _output/bin/kind/linux-amd64/kindnetd
3+
156939f66cbae4e3f857ee424cfd490086cc13de053eaa5c8502c8ae52b75dc4 _output/bin/kind/linux-arm64/kind
4+
630246bbe7e172e88fcc08f09d0d2f97a5f6122f3431ff3284f3ffa71ff60a62 _output/bin/kind/linux-arm64/kindnetd

‎projects/kubernetes-sigs/kind/patches/0001-Switch-to-AL2-base-image-for-node-image.patch‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From af841b3f8a820b38ce61fe05efd716c963bff064 Mon Sep 17 00:00:00 2001
1+
From c89d1a684f339921502ecd8f3d06bb147a10265b Mon Sep 17 00:00:00 2001
22
From: Jackson West <jgw@amazon.com>
33
Date: Thu, 29 Jun 2023 00:56:53 -0700
4-
Subject: [PATCH 1/7] Switch to AL2 base image for node image
4+
Subject: [PATCH 1/8] Switch to AL2 base image for node image
55

66
---
77
images/base/Dockerfile | 218 ++++++------------

‎projects/kubernetes-sigs/kind/patches/0002-skip-ctr-pulling-required-images-since-the-build-rem.patch‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From 23a4701eb8d6081f61c68414bbb8ee3c103d1243 Mon Sep 17 00:00:00 2001
1+
From 2a0a964208a712d18ceb14e79f328440c373e22a Mon Sep 17 00:00:00 2001
22
From: Jackson West <jgw@amazon.com>
33
Date: Sat, 2 Apr 2022 22:01:04 -0500
4-
Subject: [PATCH 2/7] skip ctr pulling required images since the build removes
4+
Subject: [PATCH 2/8] skip ctr pulling required images since the build removes
55
them anyway
66

77
---

‎projects/kubernetes-sigs/kind/patches/0003-Patch-haproxy-maxconn-value-to-avoid-ulimit-issue.patch‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From 326c89f28813f347fb47c6fc1a803286ac4604d4 Mon Sep 17 00:00:00 2001
1+
From f090606f2a4fba4a24741db40370af7ed752f90f Mon Sep 17 00:00:00 2001
22
From: Jackson West <jgw@amazon.com>
33
Date: Wed, 19 Apr 2023 12:28:28 -0500
4-
Subject: [PATCH 3/7] Patch haproxy maxconn value to avoid ulimit issue
4+
Subject: [PATCH 3/8] Patch haproxy maxconn value to avoid ulimit issue
55

66
EKS-A uses haproxy 2.5 which errors if the maxconn value
77
requires more FDs than allowed by the ulimit setting of docker.

‎projects/kubernetes-sigs/kind/patches/0004-Disable-cgroupns-private-to-fix-cluster-creation-on-.patch‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From bea2412acd3e40f97dce8ba23fd8718ab58f3ce1 Mon Sep 17 00:00:00 2001
1+
From 614562f681c605640ea42118132dda68809f1746 Mon Sep 17 00:00:00 2001
22
From: Jackson West <jgw@amazon.com>
33
Date: Sat, 19 Aug 2023 09:21:55 -0500
4-
Subject: [PATCH 4/7] Disable cgroupns=private to fix cluster creation on AL2
4+
Subject: [PATCH 4/8] Disable cgroupns=private to fix cluster creation on AL2
55

66
---
77
pkg/cluster/internal/providers/docker/provision.go | 3 ++-

‎projects/kubernetes-sigs/kind/patches/0005-TEMP-lock-containerd-and-runc-version.patch‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From 5b7a9f7376920449cc6363316b1c6e9de7b48d8b Mon Sep 17 00:00:00 2001
1+
From 49fbf65b3cfa2cc709e1e30228e50b31003f2f3a Mon Sep 17 00:00:00 2001
22
From: Jackson West <jgw@amazon.com>
33
Date: Sun, 20 Aug 2023 14:30:06 -0500
4-
Subject: [PATCH 5/7] TEMP: lock containerd and runc version
4+
Subject: [PATCH 5/8] TEMP: lock containerd and runc version
55

66
---
77
images/base/Dockerfile | 4 ++++

‎projects/kubernetes-sigs/kind/patches/0006-Use-docker_tag-file-to-fetch-Kubernetes-source-versi.patch‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From c104da44ef84a8a60b62d94241b869079d3e1fd2 Mon Sep 17 00:00:00 2001
1+
From 5c97ca01146e21541692694bcfc237bc62d3807d Mon Sep 17 00:00:00 2001
22
From: Abhay Krishna Arunachalam <arnchlm@amazon.com>
33
Date: Mon, 11 Nov 2024 17:25:26 -0800
4-
Subject: [PATCH 6/7] Use docker_tag file to fetch Kubernetes source version
4+
Subject: [PATCH 6/8] Use docker_tag file to fetch Kubernetes source version
55

66
Kind v0.25.0 introduced a change to use the `version` file in the Kubernetes server binaries tarball to
77
fetch the Kubernetes source version. But the server tarball from EKS Distro does not include this file.

projects/kubernetes-sigs/kind/patches/0007-Increase-containerd-socket-wait-timeout-for-QEMU.patch renamed to projects/kubernetes-sigs/kind/patches/0007-Increase-containerd-socket-wait-timeout-for-QEMU-emu.patch

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
From 9b8e331a25cb950f9e323ec557ffdf2e94f59c8e Mon Sep 17 00:00:00 2001
1+
From e0548602f71b7e4673762a23b532497dd2156edf Mon Sep 17 00:00:00 2001
22
From: EKS Anywhere Team <eks-anywhere@amazon.com>
33
Date: Mon, 12 Jan 2026 12:00:00 -0800
4-
Subject: [PATCH 7/7] Increase containerd socket wait timeout for QEMU
4+
Subject: [PATCH 7/8] Increase containerd socket wait timeout for QEMU
55
emulation
66

77
The default 6-second timeout (0+1+2+3) in WaitForReady() is insufficient

‎projects/kubernetes-sigs/kind/patches/0008-Disable-kubelet-fail-cgroupv1.patch‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
From 7e5ebfc87da453d4be36acf1040e11d860819ff2 Mon Sep 17 00:00:00 2001
2+
From: Prow Bot <prow@amazonaws.com>
3+
Date: Tue, 10 Feb 2026 16:34:44 -0800
4+
Subject: [PATCH 8/8] Disable kubelet fail-cgroupv1
5+
6+
kubelet fail-cgroupv1 defaults to true starting k8s 1.35
7+
Some of our ec2 instances used in e2e and presubmit instances
8+
use AL2 shipped with cgroup v1.
9+
10+
Temporarily disable this behaviour until we upgrade above instances.
11+
---
12+
pkg/cluster/internal/kubeadm/config.go | 8 ++++++++
13+
1 file changed, 8 insertions(+)
14+
15+
diff --git a/pkg/cluster/internal/kubeadm/config.go b/pkg/cluster/internal/kubeadm/config.go
16+
index a6120d32..4ceb3008 100644
17+
--- a/pkg/cluster/internal/kubeadm/config.go
18+
+++ b/pkg/cluster/internal/kubeadm/config.go
19+
@@ -245,6 +245,8 @@ nodeRegistration:
20+
node-ip: "{{ .NodeAddress }}"
21+
provider-id: "kind://{{.NodeProvider}}/{{.ClusterName}}/{{.NodeName}}"
22+
node-labels: "{{ .NodeLabels }}"
23+
+ # disable cgroup v1 failure to support nodes with cgroup v1
24+
+ fail-cgroupv1: "false"
25+
---
26+
# no-op entry that exists solely so it can be patched
27+
apiVersion: kubeadm.k8s.io/v1beta2
28+
@@ -263,6 +265,8 @@ nodeRegistration:
29+
node-ip: "{{ .NodeAddress }}"
30+
provider-id: "kind://{{.NodeProvider}}/{{.ClusterName}}/{{.NodeName}}"
31+
node-labels: "{{ .NodeLabels }}"
32+
+ # disable cgroup v1 failure to support nodes with cgroup v1
33+
+ fail-cgroupv1: "false"
34+
discovery:
35+
bootstrapToken:
36+
apiServerEndpoint: "{{ .ControlPlaneEndpoint }}"
37+
@@ -388,6 +392,8 @@ nodeRegistration:
38+
node-ip: "{{ .NodeAddress }}"
39+
provider-id: "kind://{{.NodeProvider}}/{{.ClusterName}}/{{.NodeName}}"
40+
node-labels: "{{ .NodeLabels }}"
41+
+ # disable cgroup v1 failure to support nodes with cgroup v1
42+
+ fail-cgroupv1: "false"
43+
{{ if .InitSkipPhases -}}
44+
skipPhases:
45+
{{- range $phase := .InitSkipPhases }}
46+
@@ -412,6 +418,8 @@ nodeRegistration:
47+
node-ip: "{{ .NodeAddress }}"
48+
provider-id: "kind://{{.NodeProvider}}/{{.ClusterName}}/{{.NodeName}}"
49+
node-labels: "{{ .NodeLabels }}"
50+
+ # disable cgroup v1 failure to support nodes with cgroup v1
51+
+ fail-cgroupv1: "false"
52+
discovery:
53+
bootstrapToken:
54+
apiServerEndpoint: "{{ .ControlPlaneEndpoint }}"
55+
--
56+
2.50.1 (Apple Git-155)
57+

0 commit comments

Comments
 (0)