Skip to content

Commit f3cd8f9

Browse files
kquinn1204kquinn1204
committed
TELCODOCS-2124 Ensure NUMA Resources Operator Works on HyperShift Hosted Clusters
Peer review updates
1 parent 6ab4e46 commit f3cd8f9

File tree

5 files changed

+174
-12
lines changed

5 files changed

+174
-12
lines changed

‎modules/cnf-configuring-kubelet-nro.adoc‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
//
33
// *scalability_and_performance/cnf-numa-aware-scheduling.adoc
44

5-
:_module-type: PROCEDURE
5+
:_mod-docs-content-type: PROCEDURE
66
[id="cnf-configuring-kubelet-config-nro_{context}"]
77
= Creating a KubeletConfig CR
88

@@ -41,11 +41,16 @@ spec:
4141
memory: "512Mi"
4242
topologyManagerPolicy: "single-numa-node" <5>
4343
----
44-
<1> Adjust this label to match the `machineConfigPoolSelector` in the `NUMAResourcesOperator` CR.
44+
<1> Ensure that this label matches the `machineConfigPoolSelector` setting in the `NUMAResourcesOperator` CR that you configure later in "Creating the NUMAResourcesOperator custom resource".
4545
<2> For `cpuManagerPolicy`, `static` must use a lowercase `s`.
4646
<3> Adjust this based on the CPU on your nodes.
4747
<4> For `memoryManagerPolicy`, `Static` must use an uppercase `S`.
4848
<5> `topologyManagerPolicy` must be set to `single-numa-node`.
49+
+
50+
[NOTE]
51+
====
52+
For hosted control plane clusters, the `machineConfigPoolSelector` setting does not have any functional effect. Node association is instead determined by the specified `NodePool` object. To apply a `KubeletConfig` for hosted control plane clusters, you must create a `ConfigMap` that contains the configuration, and then reference that `ConfigMap` within the `spec.config` field of a `NodePool`.
53+
====
4954

5055
.. Create the `KubeletConfig` CR by running the following command:
5156
+

‎modules/cnf-creating-nrop-cr-hosted-control-plane.adoc‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
// Module included in the following assemblies:
2+
//
3+
// *scalability_and_performance/cnf-numa-aware-scheduling.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="cnf-creating-nrop-cr-hosted-control-plane_{context}"]
7+
= Creating the NUMAResourcesOperator custom resource for {hcp}
8+
9+
After you install the NUMA Resources Operator, create the `NUMAResourcesOperator` custom resource (CR). The CR instructs the NUMA Resources Operator to install all the cluster infrastructure that is needed to support the NUMA-aware scheduler on {hcp}, including daemon sets and APIs.
10+
11+
--
12+
:FeatureName: Creating the NUMAResourcesOperator custom resource for {hcp}
13+
include::snippets/technology-preview.adoc[]
14+
--
15+
16+
.Prerequisites
17+
18+
* Install the OpenShift CLI (`oc`).
19+
* Log in as a user with `cluster-admin` privileges.
20+
* Install the NUMA Resources Operator.
21+
22+
.Procedure
23+
24+
. Export the management cluster kubeconfig file by running the following command:
25+
+
26+
[source,terminal]
27+
----
28+
$ export KUBECONFIG=<path-to-management-cluster-kubeconfig>
29+
----
30+
31+
. Find the `node-pool-name` for your cluster by running the following command:
32+
+
33+
[source,terminal]
34+
----
35+
$ oc --kubeconfig="$MGMT_KUBECONFIG" get np -A
36+
----
37+
+
38+
.Example output
39+
[source,terminal]
40+
----
41+
NAMESPACE NAME CLUSTER DESIRED NODES CURRENT NODES AUTOSCALING AUTOREPAIR VERSION UPDATINGVERSION UPDATINGCONFIG MESSAGE
42+
clusters democluster-us-east-1a democluster 1 1 False False 4.19.0 False False
43+
----
44+
+
45+
The `node-pool-name` is the `NAME` field in the output. In this example, the `node-pool-name` is `democluster-us-east-1a`.
46+
47+
. Create a YAML file named `nrop-hcp.yaml` with at least the following content:
48+
+
49+
[source,yaml]
50+
----
51+
apiVersion: nodetopology.openshift.io/v1
52+
kind: NUMAResourcesOperator
53+
metadata:
54+
name: numaresourcesoperator
55+
spec:
56+
nodeGroups:
57+
- poolName: democluster-us-east-1a <1>
58+
----
59+
+
60+
<1> The `poolName` is the `node-pool-name` retrieved in step 2.
61+
62+
. On the management cluster, run the following command to list the available secrets:
63+
+
64+
[source,terminal]
65+
----
66+
$ oc get secrets -n clusters
67+
----
68+
+
69+
.Example output
70+
[source,terminal]
71+
----
72+
NAME TYPE DATA AGE
73+
builder-dockercfg-25qpp kubernetes.io/dockercfg 1 128m
74+
default-dockercfg-mkvlz kubernetes.io/dockercfg 1 128m
75+
democluster-admin-kubeconfig Opaque 1 127m
76+
democluster-etcd-encryption-key Opaque 1 128m
77+
democluster-kubeadmin-password Opaque 1 126m
78+
democluster-pull-secret Opaque 1 128m
79+
deployer-dockercfg-8lfpd kubernetes.io/dockercfg 1 128m
80+
----
81+
82+
. Extract the `kubeconfig` file for the hosted cluster by running the following command:
83+
+
84+
[source,terminal]
85+
----
86+
$ oc get secret <SECRET_NAME> -n clusters -o jsonpath='{.data.kubeconfig}' | base64 -d > hosted-cluster-kubeconfig
87+
----
88+
+
89+
.Example
90+
[source,terminal]
91+
----
92+
$ oc get secret democluster-admin-kubeconfig -n clusters -o jsonpath='{.data.kubeconfig}' | base64 -d > hosted-cluster-kubeconfig
93+
----
94+
95+
. Export the hosted cluster `kubeconfig` file by running the following command:
96+
+
97+
[source,terminal]
98+
----
99+
$ export HC_KUBECONFIG=<path_to_hosted-cluster-kubeconfig>
100+
----
101+
102+
. Create the `NUMAResourcesOperator` CR by running the following command on the hosted cluster:
103+
+
104+
[source,terminal]
105+
----
106+
$ oc create -f nrop-hcp.yaml
107+
----
108+
109+
.Verification
110+
111+
. Verify that the NUMA Resources Operator deployed successfully by running the following command:
112+
+
113+
[source,terminal]
114+
----
115+
$ oc get numaresourcesoperators.nodetopology.openshift.io
116+
----
117+
+
118+
.Example output
119+
[source,terminal]
120+
----
121+
NAME AGE
122+
numaresourcesoperator 27s
123+
----
124+
125+
. After a few minutes, run the following command to verify that the required resources deployed successfully:
126+
+
127+
[source,terminal]
128+
----
129+
$ oc get all -n openshift-numaresources
130+
----
131+
+
132+
.Example output
133+
[source,terminal]
134+
----
135+
NAME READY STATUS RESTARTS AGE
136+
pod/numaresources-controller-manager-7d9d84c58d-qk2mr 1/1 Running 0 12m
137+
pod/numaresourcesoperator-democluster-7d96r 2/2 Running 0 97s
138+
pod/numaresourcesoperator-democluster-crsht 2/2 Running 0 97s
139+
pod/numaresourcesoperator-democluster-jp9mw 2/2 Running 0 97s
140+
----

‎modules/cnf-deploying-the-numa-aware-scheduler.adoc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,11 @@ spec:
3535
----
3636
$ oc create -f nro-scheduler.yaml
3737
----
38+
+
39+
[NOTE]
40+
====
41+
In a hosted control plane cluster, run this command on the hosted control plane node.
42+
====
3843

3944
. After a few seconds, run the following command to confirm the successful deployment of the required resources:
4045
+

‎modules/cnf-sample-single-numa-policy-from-pp.adoc‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
//
33
// *scalability_and_performance/cnf-numa-aware-scheduling.adoc
44

5-
:_module-type: REFERENCE
5+
:_mod-docs-content-type: REFERENCE
66
[id="cnf-sample-performance-policy_{context}"]
77
= Sample performance profile
88

@@ -32,5 +32,10 @@ spec:
3232
realTime: true
3333
----
3434

35-
<1> This should match the `MachineConfigPool` that you want to configure the NUMA Resources Operator on. For example, you might have created a `MachineConfigPool` named `worker-cnf` that designates a set of nodes that run telecommunications workloads.
36-
<2> The `topologyPolicy` must be set to `single-numa-node`. Ensure that this is the case by setting the `topology-manager-policy` argument to `single-numa-node` when running the PPC tool.
35+
<1> This value must match the `MachineConfigPool` value that you want to configure the NUMA Resources Operator on. For example, you might create a `MachineConfigPool` object named `worker-cnf` that designates a set of nodes that run telecommunications workloads. The value for `MachineConfigPool` must match the `machineConfigPoolSelector` value in the `NUMAResourcesOperator` CR that you configure later in "Creating the NUMAResourcesOperator custom resource".
36+
<2> Ensure that the `topologyPolicy` field is set to `single-numa-node` by setting the `topology-manager-policy` argument to `single-numa-node` when you run the PPC tool.
37+
+
38+
[NOTE]
39+
====
40+
For hosted control plane clusters, the `machineConfigPoolSelector` does not have any functional effect. Node association is instead determined by the specified `NodePool` object.
41+
====

‎scalability_and_performance/cnf-numa-aware-scheduling.adoc‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -32,13 +32,7 @@ include::modules/cnf-installing-numa-resources-operator-cli.adoc[leveloffset=+2]
3232

3333
include::modules/cnf-installing-numa-resources-operator-console.adoc[leveloffset=+2]
3434

35-
include::modules/cnf-scheduling-numa-aware-workloads-overview.adoc[leveloffset=+1]
36-
37-
include::modules/cnf-creating-nrop-cr.adoc[leveloffset=+2]
38-
39-
include::modules/cnf-deploying-the-numa-aware-scheduler.adoc[leveloffset=+2]
40-
41-
include::modules/cnf-configuring-single-numa-policy.adoc[leveloffset=+2]
35+
include::modules/cnf-configuring-single-numa-policy.adoc[leveloffset=+1]
4236

4337
[role="_additional-resources"]
4438
.Additional resources
@@ -51,6 +45,19 @@ include::modules/cnf-sample-single-numa-policy-from-pp.adoc[leveloffset=+2]
5145

5246
include::modules/cnf-configuring-kubelet-nro.adoc[leveloffset=+2]
5347

48+
include::modules/cnf-scheduling-numa-aware-workloads-overview.adoc[leveloffset=+1]
49+
50+
include::modules/cnf-creating-nrop-cr.adoc[leveloffset=+2]
51+
52+
include::modules/cnf-creating-nrop-cr-hosted-control-plane.adoc[leveloffset=+2]
53+
54+
[role="_additional-resources"]
55+
.Additional resources
56+
57+
* xref:../scalability_and_performance/cnf-tuning-low-latency-hosted-cp-nodes-with-perf-profile.adoc#cnf-create-performance-profiles-hosted-cp[Creating a performance profile for hosted control planes]
58+
59+
include::modules/cnf-deploying-the-numa-aware-scheduler.adoc[leveloffset=+2]
60+
5461
include::modules/cnf-scheduling-numa-aware-workloads.adoc[leveloffset=+2]
5562

5663
include::modules/cnf-configuring-node-groups-for-the-numaresourcesoperator.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)