Merge pull request #101218 from dfitzmau/OCPBUGS-61836

OCPBUGS-61836: Updated the br-ex NNCP example's ip values

Author: dfitzmau

installing/installing_bare_metal/upi/installing-bare-metal.adoc

@@ -6,16 +6,12 @@ include::_attributes/common-attributes.adoc[]
 
 toc::[]
 
-In {product-title} {product-version}, you can install a cluster on
-bare metal infrastructure that you provision.
+In {product-title} {product-version}, you can install a cluster on bare-metal infrastructure that you provision.
 
 [IMPORTANT]
 ====
-While you might be able to follow this procedure to deploy a cluster on
-virtualized or cloud environments, you must be aware of additional
-considerations for non-bare metal platforms. Review the information in the
-link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms]
-before you attempt to install an {product-title} cluster in such an environment.
+While you might be able to follow this procedure to deploy a cluster on virtualized or cloud environments, you must be aware of additional
+considerations for non-bare-metal platforms. Review the information in the link:https://access.redhat.com/articles/4207611[guidelines for deploying {product-title} on non-tested platforms] before you attempt to install an {product-title} cluster in such an environment.
 ====
 
 == Prerequisites
@@ -29,6 +25,7 @@ before you attempt to install an {product-title} cluster in such an environment.
 Be sure to also review this site list if you are configuring a proxy.
 ====
 
+// Internet access for OpenShift Container Platform
 include::modules/cluster-entitlements.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
@@ -44,8 +41,10 @@ of the required machines.
 
 This section describes the requirements for deploying {product-title} on user-provisioned infrastructure.
 
+// Required machines for cluster installation
 include::modules/installation-machine-requirements.adoc[leveloffset=+2]
 
+// Minimum resource requirements for cluster installation
 include::modules/installation-minimum-resource-requirements.adoc[leveloffset=+2]
 
 [role="_additional-resources"]
@@ -86,6 +85,7 @@ include::modules/installation-dns-user-infra.adoc[leveloffset=+2]
 
 * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure]
 
+// Load balancing requirements for user-provisioned infrastructure
 include::modules/installation-load-balancing-user-infra.adoc[leveloffset=+2]
 
 // Creating a manifest object that includes a customized `br-ex` bridge
@@ -104,6 +104,7 @@ include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[lev
 // Enabling OVS balance-slb mode for your cluster
 include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
 
+// Preparing the user-provisioned infrastructure
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
@@ -118,6 +119,7 @@ include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-user-provisioned-validating-dns_installing-bare-metal[Validating DNS resolution for user-provisioned infrastructure]
 * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure]
 
+// Validating DNS resolution for user-provisioned infrastructure
 include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
@@ -126,23 +128,28 @@ include::modules/installation-user-provisioned-validating-dns.adoc[leveloffset=+
 * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-dns-user-infra_installing-bare-metal[User-provisioned DNS requirements]
 * xref:../../../installing/installing_bare_metal/upi/installing-bare-metal.adoc#installation-load-balancing-user-infra_installing-bare-metal[Load balancing requirements for user-provisioned infrastructure]
 
+// Generating a key pair for cluster node SSH access
 include::modules/ssh-agent-using.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
 .Additional resources
 
 * xref:../../../support/troubleshooting/verifying-node-health.adoc#verifying-node-health[Verifying node health]
 
+// Obtaining the installation program
 include::modules/installation-obtaining-installer.adoc[leveloffset=+1]
 
+// Installing the OpenShift CLI
 include::modules/cli-installing-cli.adoc[leveloffset=+1]
 
+// Manually creating the installation configuration file
 include::modules/installation-initializing-manual.adoc[leveloffset=+1]
 
 [role="_additional-resources"]
 .Additional resources
 * xref:../../../installing/installing_bare_metal/upi/installation-config-parameters-bare-metal.adoc#installation-config-parameters-bare-metal[Installation configuration parameters for bare metal]
 
+// Sample install-config.yaml file for bare metal
 include::modules/installation-bare-metal-config-yaml.adoc[leveloffset=+2]
 
 [role="_additional-resources"]

modules/creating-manifest-file-customized-br-ex-bridge.adoc

@@ -19,7 +19,9 @@ As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex`
 endif::postinstall-bare-metal[]
 
 ifdef::postinstall-bare-metal[]
-As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.
+As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `NodeNetworkConfigurationPolicy` (NNCP) custom resource (CR) that includes an NMState configuration file. 
+
+The Kubernetes NMState Operator uses the NMState configuration file to create a customized `br-ex` bridge network configuration on each node in your cluster.
 
 [IMPORTANT]
 ====
@@ -87,11 +89,11 @@ ifndef::postinstall-bare-metal[]
 [source,yaml]
 ----
 interfaces:
-- name: enp2s0 <1>
-  type: ethernet <2>
-  state: up <3>
+- name: enp2s0
+  type: ethernet
+  state: up
   ipv4:
-    enabled: false <4>
+    enabled: false
   ipv6:
     enabled: false
 - name: br-ex
@@ -107,7 +109,7 @@ interfaces:
     options:
       mcast-snooping-enable: true
     port:
-    - name: enp2s0 <5>
+    - name: enp2s0
     - name: br-ex
 - name: br-ex
   type: ovs-interface
@@ -116,27 +118,33 @@ interfaces:
   ipv4:
     enabled: true
     dhcp: true
-    auto-route-metric: 48 <6>
+    auto-route-metric: 48
   ipv6:
     enabled: true
     dhcp: true
     auto-route-metric: 48
 # ...
 ----
-<1> Name of the interface.
-<2> The type of ethernet.
-<3> The requested state for the interface after creation.
-<4> Disables IPv4 and IPv6 in this example.
-<5> The node NIC to which the bridge attaches.
-<6> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
++
+where:
++
+`interfaces.name`:: Name of the interface.
+`interfaces.type`:: The type of ethernet.
+`interfaces.state`:: The requested state for the interface after creation.
+`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
+`port.name`:: The node NIC to which the bridge attaches.
+`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
 
 . Use the `cat` command to base64-encode the contents of the NMState configuration:
 +
 [source,terminal]
 ----
-$ cat <nmstate_configuration>.yaml | base64 <1>
+$ cat <nmstate_configuration>.yml | base64
 ----
-<1> Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.
++
+where:
++
+`<nmstate_configuration>`:: Replace `<nmstate_configuration>` with the name of your NMState resource YAML file.
 
 . Create a `MachineConfig` manifest file and define a customized `br-ex` bridge network configuration analogous to the following example:
 +
@@ -147,28 +155,31 @@ kind: MachineConfig
 metadata:
   labels:
     machineconfiguration.openshift.io/role: worker
-  name: 10-br-ex-worker <1>
+  name: 10-br-ex-worker
 spec:
   config:
     ignition:
       version: 3.2.0
     storage:
       files:
       - contents:
-          source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration> <2>
+          source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
         mode: 0644
         overwrite: true
-        path: /etc/nmstate/openshift/worker-0.yml <3>
+        path: /etc/nmstate/openshift/worker-0.yml
       - contents:
           source: data:text/plain;charset=utf-8;base64,<base64_encoded_nmstate_configuration>
         mode: 0644
         overwrite: true
-        path: /etc/nmstate/openshift/worker-1.yml <3>
+        path: /etc/nmstate/openshift/worker-1.yml
 # ...
 ----
-<1> The name of the policy.
-<2> Writes the encoded base64 information to the specified path.
-<3> For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. The `.yaml` extension does not work when specifying the short hostname, `hostname -s`, path for each node or all nodes in the `MachineConfig` manifest file.
++
+where:
++
+`metadata.name`:: The name of the policy.
+`contents.source`:: Writes the encoded base64 information to the specified path.
+`path`:: For each node in your cluster, specify the hostname path to your node and the base-64 encoded Ignition configuration file data for the machine type. The `worker` role is the default role for nodes in your cluster. You must use the `.yml` extension for configuration files, such as `$(hostname -s).yml` when specifying the short hostname path for each node or all nodes in the `MachineConfig` manifest file.
 +
 If you have a single global configuration specified in an `/etc/nmstate/openshift/cluster.yml` configuration file that you want to apply to all nodes in your cluster, you do not need to specify the short hostname path for each node, such as `/etc/nmstate/openshift/<node_hostname>.yml`. For example:
 +
@@ -192,32 +203,29 @@ $ oc apply -f <machine_config>.yml
 endif::postinstall-bare-metal[]
 
 ifdef::postinstall-bare-metal[]
-* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. Depending on your needs, ensure that you set a masquerade IP for either the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters. Always include a masquerade IP address in the NNCP CR and this address must match an in-use IP address block.
+* Create a `NodeNetworkConfigurationPolicy` (NNCP) CR and define a customized `br-ex` bridge network configuration. The `br-ex` NNCP CR must include the OVN-Kubernetes masquerade IP address and subnet of your network. The example NNCP CR includes default values in the `ipv4.address.ip` and `ipv6.address.ip` parameters. You can set the masquerade IP address in the `ipv4.address.ip`, `ipv6.address.ip`, or both parameters.
 +
 [IMPORTANT]
 ====
-As a post-installation task, you can configure most parameters for a customized `br-ex` bridge that you defined in an existing NNCP CR, except for the primary IP address of the customized `br-ex` bridge.
-
-If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
+As a post-installation task, you cannot change the primary IP address of the customized `br-ex` bridge. If you want to convert your single-stack cluster network to a dual-stack cluster network, you can add or change a secondary IPv6 address in the NNCP CR, but the existing primary IP address cannot be changed.
 ====
 +
-.Example of an NNCP CR that sets IPv6 and IPv4 masquerade IP addresses
 [source,yaml]
 ----
 apiVersion: nmstate.io/v1
 kind: NodeNetworkConfigurationPolicy
 metadata:
-  name: worker-0-br-ex <1>
+  name: worker-0-br-ex
 spec:
   nodeSelector:
     kubernetes.io/hostname: worker-0
     desiredState:
     interfaces:
-    - name: enp2s0 <2>
-      type: ethernet <3>
-      state: up <4>
+    - name: enp2s0
+      type: ethernet
+      state: up
       ipv4:
-        enabled: false <5>
+        enabled: false
       ipv6:
         enabled: false
     - name: br-ex
@@ -233,7 +241,7 @@ spec:
         options:
           mcast-snooping-enable: true
         port:
-        - name: enp2s0 <6>
+        - name: enp2s0
         - name: br-ex
     - name: br-ex
       type: ovs-interface
@@ -242,26 +250,30 @@ spec:
       ipv4:
         enabled: true
         dhcp: true
-        auto-route-metric: 48 <7>
+        auto-route-metric: 48
         address:
-        - ip: "169.254.169.2"
-          prefix-length: 29
+        - ip: "169.254.0.2"
+          prefix-length: 17
       ipv6:
         enabled: true
         dhcp: true
         auto-route-metric: 48
         address:
         - ip: "fd69::2"
-        prefix-length: 125
+        prefix-length: 112
 # ...
 ----
-<1> Name of the policy.
-<2> Name of the interface.
-<3> The type of ethernet.
-<4> The requested state for the interface after creation.
-<5> Disables IPv4 and IPv6 in this example.
-<6> The node NIC to which the bridge is attached.
-<7> Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
++
+where:
++
+`metadata.name`:: Name of the policy.
+`interfaces.name`:: Name of the interface.
+`interfaces.type`:: The type of ethernet.
+`interfaces.state`:: The requested state for the interface after creation.
+`ipv4.enabled`:: Disables IPv4 and IPv6 in this example.
+`port.name`:: The node NIC to which the bridge is attached.
+`address.ip`:: Shows the default IPv4 and IPv6 IP addresses. Ensure that you set the masquerade IPv4 and IPv6 IP addresses of your network. 
+`auto-route-metric`:: Set the parameter to `48` to ensure the `br-ex` default route always has the highest precedence (lowest metric). This configuration prevents routing conflicts with any other interfaces that are automatically configured by the `NetworkManager` service.
 endif::postinstall-bare-metal[]
 
 .Next steps

modules/ipi-install-additional-install-config-parameters.adoc

@@ -136,7 +136,7 @@ a| `provisioningNetworkInterface` |  | The name of the network interface on node
 
 | `apiVIPs` | a| (Optional) The virtual IP address for Kubernetes API communication.
 
-You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `api.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
+You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `apiVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `api.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
 
 [NOTE]
 ====
@@ -148,7 +148,7 @@ Before {product-title} 4.12, the cluster installation program only accepted an I
 
 | `ingressVIPs` | a| (Optional) The virtual IP address for ingress traffic.
 
-You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. The primary IP address must be from the IPv4 network when using dual stack networking. If not set, the installation program uses `test.apps.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
+You must either provide this setting in the `install-config.yaml` file as a reserved IP from the `MachineNetwork` parameter or preconfigured in the DNS so that the default name resolves correctly. Use the virtual IP address and not the FQDN when adding a value to the `ingressVIPs` configuration setting in the `install-config.yaml` file. For dual-stack networking, the primary IP address can be either an IPv4 network or an IPv6 network. If not set, the installation program uses `test.apps.<cluster_name>.<base_domain>` to derive the IP address from the DNS.
 
 [NOTE]
 ====

networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc

@@ -13,7 +13,7 @@ For more information about how to install the NMState Operator, see xref:../../n
 
 [IMPORTANT]
 ====
-You cannot provide any configuration that modifies the br-ex bridge, an OVN-Kubernetes-managed Open vSwitch bridge. However, you can configure a customized br-ex bridge.
+You cannot modify an existing `br-ex` bridge, an OVN-Kubernetes-managed Open vSwitch bridge, or any interfaces, bonds, VLANs, and so on that associate with the `br-ex` bridge. However, you can configure a customized br-ex bridge.
 
 For more information, see "Creating a manifest object that includes a customized br-ex bridge" in the _Deploying installer-provisioned clusters on bare metal_ document or the _Installing a user-provisioned cluster on bare metal_ document.
 ====