Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ func main() {
var enableWebhook bool
var metricsSecure bool
var metricsCertDir string
var leaderElectionNamespace string
flag.StringVar(&metricsAddr, "metrics-bind-address", "0", "The address the metrics endpoint binds to. "+
"Use :8443 for HTTPS or :8080 for HTTP, or leave as 0 to disable the metrics service.")
flag.BoolVar(&metricsSecure, "metrics-secure", false,
Expand All @@ -78,6 +79,7 @@ func main() {
"Enabling this will ensure there is only one active controller manager.")
flag.BoolVar(&enableWebhook, "enable-webhook", false,
"Enable validation webhook. Requires TLS certificates to be configured.")
flag.StringVar(&leaderElectionNamespace, "leader-election-namespace", "", "The namespace where the leader election resource will be created.")

opts := zap.Options{
Development: true,
Expand All @@ -102,11 +104,12 @@ func main() {
}

mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: scheme,
Metrics: metricsServerOptions,
HealthProbeBindAddress: probeAddr,
LeaderElection: enableLeaderElection,
LeaderElectionID: "ba65f13e.readiness.node.x-k8s.io",
Scheme: scheme,
Metrics: metricsServerOptions,
HealthProbeBindAddress: probeAddr,
LeaderElection: enableLeaderElection,
LeaderElectionID: "ba65f13e.readiness.node.x-k8s.io",
LeaderElectionNamespace: leaderElectionNamespace,
})
if err != nil {
setupLog.Error(err, "unable to start manager")
Expand Down
43 changes: 38 additions & 5 deletions docs/book/src/user-guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,21 +76,44 @@ kubectl apply -k config/default

You can enable optional components (Metrics, TLS, Webhook) by creating a `kustomization.yaml` that includes the relevant components from the `config/` directory. For reference on how these components can be combined, see the `deploy-with-metrics`, `deploy-with-tls`, `deploy-with-webhook`, and `deploy-full` targets in the projects [`Makefile`](https://github.com/kubernetes-sigs/node-readiness-controller/blob/main/Makefile).

---
### Option 3: Deploy as a Static Pod (Control Plane)

Running the controller as a **Static Pod** on control-plane nodes is useful for self-managed clusters (e.g., `kubeadm`) where you want the controller to be available alongside core components like the API server.

Refer to the `examples/static-pod/node-readiness-controller.yaml` for a detailed
example on deploying the controller as a static pod in a kind cluster.

#### Deployment Steps

1. **Prepare the Manifest**:
Refer the example manifest in
`examples/static-pod/node-readiness-controller.yaml`. This manifest handles kubeconfig with a `initContainer` and necessary flags for leader election.

2. **Deploy to Nodes**:
Use Ansible / Terraform to copy the manifest to the `/etc/kubernetes/manifests/` directory on each control-plane node. The Kubelet will automatically detect and start the pod.

3. **Install CRDs**:
```sh
kubectl apply -k config/crd
```
This is typically handled via a bootstrap script or post-install job in a `kubeadm` setup.

---
## Verification

After installation, verify that the controller is running successfully.
After installation, verify that the controller is running successfully.

> **Note**: Replace `${NAMESPACE}` with the namespace where the controller is deployed (typically `nrr-system` for standard deployments, or `kube-system` for static pods).

1. **Check Pod Status**:
```sh
kubectl get pods -n nrr-system
kubectl get pods -n ${NAMESPACE} -l component=node-readiness-controller
```
You should see a pod named `nrr-controller-manager-...` in `Running` status.
You should see the controller pods in `Running` status.

2. **Check Logs**:
```sh
kubectl logs -n nrr-system -l control-plane=controller-manager
kubectl logs -n ${NAMESPACE} -l component=node-readiness-controller
```
Look for "Starting EventSource" or "Starting Controller" messages indicating the manager is active.

Expand All @@ -99,6 +122,13 @@ After installation, verify that the controller is running successfully.
kubectl get crd nodereadinessrules.readiness.node.x-k8s.io
```

4. **Verify High Availability**:
In an HA cluster, verify that one instance has acquired the leader lease:
```sh
# The lease namespace should match the controller's namespace (configured via --leader-election-namespace)
kubectl get lease -n ${NAMESPACE} ba65f13e.readiness.node.x-k8s.io
```

## Uninstallation

> **IMPORTANT**: Follow this order to avoid "stuck" resources.
Expand All @@ -123,6 +153,9 @@ The controller uses a **finalizer** (`readiness.node.x-k8s.io/cleanup-taints`) o

# OR if using Kustomize
kubectl delete -k config/default

# OR if using Static Pods
# Remove the manifest from /etc/kubernetes/manifests/ on all control-plane nodes
```

3. **Uninstall CRDs** (Optional):
Expand Down
35 changes: 35 additions & 0 deletions examples/static-pod/kind-static-pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: nrr-static
nodes:
- role: control-plane
extraMounts:
- hostPath: ./examples/static-pod/node-readiness-controller.yaml
containerPath: /etc/kubernetes/manifests/node-readiness-controller.yaml
readOnly: true
- role: control-plane
extraMounts:
- hostPath: ./examples/static-pod/node-readiness-controller.yaml
containerPath: /etc/kubernetes/manifests/node-readiness-controller.yaml
readOnly: true
- role: control-plane
extraMounts:
- hostPath: ./examples/static-pod/node-readiness-controller.yaml
containerPath: /etc/kubernetes/manifests/node-readiness-controller.yaml
readOnly: true
- role: worker
kubeadmConfigPatches:
- |
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "reserved-for=worker"
register-with-taints: "readiness.k8s.io/NetworkReady=pending:NoSchedule"
- role: worker
kubeadmConfigPatches:
- |
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "reserved-for=worker"
register-with-taints: "readiness.k8s.io/NetworkReady=pending:NoSchedule"
54 changes: 54 additions & 0 deletions examples/static-pod/node-readiness-controller.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
apiVersion: v1
kind: Pod
metadata:
name: node-readiness-controller
namespace: kube-system
labels:
component: node-readiness-controller
tier: control-plane
spec:
# Init container to prepare a readable kubeconfig for the non-root manager.
initContainers:
- name: prepare-kubeconfig
image: busybox:latest
imagePullPolicy: IfNotPresent
command:
- sh
- -c
- |
cp /etc/kubernetes/admin.conf /tmp/kubeconfig/admin.conf
chmod 644 /tmp/kubeconfig/admin.conf
volumeMounts:
- name: host-kubeconfig
mountPath: /etc/kubernetes/admin.conf
readOnly: true
- name: nrc-kubeconfig
mountPath: /tmp/kubeconfig
containers:
- name: manager
image: controller:latest
imagePullPolicy: IfNotPresent
command:
- /manager
args:
- --kubeconfig=/etc/kubernetes/nrc/admin.conf
- --leader-elect=true
- --leader-election-namespace=kube-system
- --metrics-bind-address=:8082
- --health-probe-bind-address=:8081
- --zap-log-level=info
securityContext:
runAsUser: 65532
volumeMounts:
- name: nrc-kubeconfig
mountPath: /etc/kubernetes/nrc
readOnly: true
volumes:
- name: host-kubeconfig
hostPath:
path: /etc/kubernetes/admin.conf
type: File
- name: nrc-kubeconfig
emptyDir: {}
hostNetwork: true
priorityClassName: system-node-critical
Loading