OSDOCS-14356-New: Added bond best practices info to networking docs

Author: dfitzmau

_topic_maps/_topic_map.yml

@@ -1739,6 +1739,8 @@ Topics:
     File: verifying-connectivity-endpoint
   - Name: Changing the cluster network MTU
     File: changing-cluster-network-mtu
+  - Name: Network bonding consideration
+    File: network-bonding-considerations
   - Name: Using Stream Control Transmission Protocol
     File: using-sctp
   - Name: Associating secondary interfaces metrics to network attachments

installing/installing_bare_metal/bare-metal-postinstallation-configuration.adoc

@@ -34,9 +34,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 
 * xref:../../installing/installing_bare_metal/bare-metal-expanding-the-cluster.adoc#bare-metal-expanding-the-cluster[Expanding the cluster]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 // Services for a user-managed load balancer
 include::modules/nw-osp-services-external-load-balancer.adoc[leveloffset=+1]
 

installing/installing_bare_metal/ipi/ipi-install-installation-workflow.adoc

@@ -38,9 +38,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 // Establishing communication between subnets
 include::modules/ipi-install-establishing-communication-between-subnets.adoc[leveloffset=+1]
 

installing/installing_bare_metal/upi/installing-bare-metal-network-customizations.adoc

@@ -79,9 +79,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]

installing/installing_bare_metal/upi/installing-bare-metal.adoc

@@ -100,9 +100,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]

installing/installing_bare_metal/upi/installing-restricted-networks-bare-metal.adoc

@@ -94,9 +94,6 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset
 // Scale each machine set to compute nodes
 include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2]
 
-// Enabling OVS balance-slb mode for your cluster
-include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1]
-
 include::modules/installation-infrastructure-user-infra.adoc[leveloffset=+1]
 
 [role="_additional-resources"]

modules/installation-network-user-infra.adoc

@@ -87,9 +87,17 @@ endif::ibm-z[]
 ifndef::ibm-z[]
 During the initial boot, the machines require an IP address configuration that is set either through a DHCP server or statically by providing the required boot options. After a network connection is established, the machines download their Ignition config files from an HTTP or HTTPS server. The Ignition config files are then used to set the exact state of each machine. The Machine Config Operator completes more changes to the machines, such as the application of new certificates or keys, after installation.
 
+Use a DHCP server for the long-term management of the machines for your cluster. Ensure that the DHCP server is configured to provide persistent IP addresses, DNS server information, and hostnames to the cluster machines. As a cluster administrator, ensure that you reserve the following IP addresses components that interact with the DHCP server:
+
+* Two unique virtual IP (VIP) addresses. One VIP address for the API endpoint and one VIP address for the wildcard ingress endpoint.
+* One IP address for the provisioner node. 
+* An IP address for each control plane node.
+* An IP address for each compute node.
+If you have multiple network interfaces that interact with a bonded interface, reserve the same IP addresses for these multiple network interfaces so to ensure better load balancing, fault tolerance, and bandwidth capabilites for your cluster network infrastructure. 
+
 [NOTE]
 ====
-* It is recommended to use a DHCP server for long-term management of the cluster machines. Ensure that the DHCP server is configured to provide persistent IP addresses, DNS server information, and hostnames to the cluster machines.
+* Use a DHCP server for long-term management of the cluster machines. Ensure that the DHCP server is configured to provide persistent IP addresses, DNS server information, and hostnames to the cluster machines.
 
 * If a DHCP service is not available for your user-provisioned infrastructure, you can instead provide the IP networking configuration and the address of the DNS server to the nodes at {op-system} install time. These can be passed as boot arguments if you are installing from an ISO image. See the _Installing {op-system} and starting the {product-title} bootstrap process_ section for more information about static IP provisioning and advanced networking options.
 ====

modules/installation-user-infra-machines-static-network.adoc

@@ -220,7 +220,20 @@ ifndef::ibm-z-kvm[]
 [discrete]
 === Bonding multiple network interfaces to a single interface
 
-Optional: You can bond multiple network interfaces to a single interface by using the `bond=` option. Refer to the following examples:
+As an optional configuration, you can bond multiple network interfaces to a single interface by using the `bond=` option. To apply this configuration to your cluster, complete the procedure steps for each node that runs on your cluster.
+
+[IMPORTANT]
+====
+A bonding mode specifies the policy for how bond interfaces are used during network transmission. If your network configuration includes an Open vSwitch (OVS) interface and you enabled `active-backup` bond mode, you must specify a Media Access Control (MAC) address failover. This configuration prevents node communication issues with the bond interfaces, such as `eno1f0` and `eno2f0`.
+
+The following list details supported values for the `fail_over_mac` parameter:
+
+* `0`: Specifies the `none` value and this is the default value that disables MAC address failover so that all nodes receive the same MAC address as the bond interface. With this setting, packets might be sent to inactive nodes.
+* `1`: Specifies the `active` value and sets the MAC address of the primary bond interface to always remain the same as active nodes. If during a failover, the MAC address of a node changes, the MAC address of the bond interface changes to match the new MAC address of the node.
+* `2`: Specifies the `follow` value so that during a failover, an active node gets the MAC address of the bond interface and a formerly active node receives the MAC address of the newly active node.
+====
+
+.Procedure
 
 * The syntax for configuring a bonded interface is: `bond=<name>[:<network_interfaces>][:options]`
 +
@@ -235,7 +248,7 @@ information for the bonded interface.
 [source,terminal]
 ----
 bond=bond0:em1,em2:mode=active-backup
-ip=bond0:dhcp
+fail_over_mac=1
 ----
 
 ** To configure the bonded interface to use a static IP address, enter the specific IP address you want and related information. For example:
@@ -266,7 +279,6 @@ Optional: You can configure VLANs on bonded interfaces by using the `vlan=` para
 
 [source,terminal]
 ----
-ip=bond0.100:dhcp
 bond=bond0:em1,em2:mode=active-backup
 vlan=bond0.100:bond0
 ----
@@ -287,9 +299,9 @@ ifndef::ibm-z[]
 [discrete]
 === Bonding multiple SR-IOV network interfaces to a dual port NIC interface
 
-Optional: You can bond multiple SR-IOV network interfaces to a dual port NIC interface by using the `bond=` option.
+As an optional configuration, you can bond multiple SR-IOV network interfaces to a dual port NIC interface by using the `bond=` option.
 
-On each node, you must perform the following tasks:
+.Procedure
 
 ifndef::installing-ibm-power[]
 . Create the SR-IOV virtual functions (VFs) following the guidance in link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_virtualization/managing-virtual-devices_configuring-and-managing-virtualization#managing-sr-iov-devices_managing-virtual-devices[Managing SR-IOV devices]. Follow the procedure in the "Attaching SR-IOV networking devices to virtual machines" section.
@@ -313,7 +325,6 @@ The following examples illustrate the syntax you must use:
 [source,terminal]
 ----
 bond=bond0:eno1f0,eno2f0:mode=active-backup
-ip=bond0:dhcp
 ----
 
 ** To configure the bonded interface to use a static IP address, enter the specific IP address you want and related information. For example:

modules/nw-ovs-bonding.adoc

@@ -0,0 +1,33 @@
+// Module included in the following assemblies:
+//
+// * networking/advanced_networking/network-bonding-considerations.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="nw-ovs-bonding_{context}"]
+= Open vSwitch (OVS) bonding
+
+With an OVS bonding configuration on your network, each physical interface acts as a port and connects to a specific bond. A bond then connects to a virtual switch or an OVS bridge. This connection layout provides increased bandwidth and fault tolerance capabilities for traffic that runs on your network. 
+
+Consider the following architectural layout for OVS bridges that interact with OVS interfaces:
+
+* The bridge MAC address is used for local communication.
+* The physical MAC addresses of physical interfaces do not handle traffic.
+* OVS handles all MAC address management at the OVS bridge level.
+
+This layout simplies bond interface management as bonds acts as data paths where MAC address managements is centralized at the OVS bridge level.
+
+You can choose the following OVS bonding modes for network:
+
+* `active-backup` mode provides link aggregation capabilities for your network, where one physical interface acts as the active port while other physical interfaces act as standby ports. This mode provides fault tolerance connections for your network.
+* `kernel-bonding` mode is a built-in Linux kernel function where link aggregation can exist among mutliple Ethernet interfaces to create a single logical physical interface. This mode does not provide the same level of customization as supported OVS mode, such as `balance-slb` mode.
+* `balance-slb` mode, where an interface provides source load balancing (SLB) capabilities for a cluster that runs virtualization workloads. The interface can act independently without needing to communicate with a network switch.
+
+For `kernel-bonding` mode, the bond interfaces exist outside, which means they are not in the data path, of the bridge interface. Network traffic in this mode is not sent or received on the bond interface port but instead requires additional bridging capabilities for MAC address assignment at the Kernel level. For `active-backup` and `balance-slb` modes, the bond interfaces exist in the same data path as the OVS bridge interface, so the OVS bridge can manage bonding logic instead of the physcial interfaces manages traffic. 
+
+Enabling `balance-slb` mode for an OVS bonding configuration provides source Media Access Control (MAC) hash-based load balancing capabilities to your network. With this mode, the source MAC hash is processed as a hash function that takes the MAC address as input. Outputted hash information determines the physical interface that acts as the bond. Consider enabling this mode for an advanced network configuration that has multiple source IP addresses and ports. 
+
+Consider that an OVS bond with `balance-slb` mode enabled might experience issues if the bond forwards unknown unicast traffic from one physical network interface controller (NIC) into the phsycial network through another NIC. Such a situation can result in an Layer 2 loop, or _bridge loop_, that in turn causes MAC flapping, where the same MAC address exists in multiple network locations for a period of time, for physical switches that exist in the network infrastructure.
+
+This behavior is expected as a remote switch does not learn the MAC address for the destination of a unicast packet and this causes the packet to exist on all links available on the SLB bond configuration. As a workaround for this issue, you can set the bond to `active-backup` mode during MAC address assignment and then switch the bond to use `balance-slb` mode.
+
+

modules/nw-understanding-networking-service-to-pod.adoc

@@ -28,7 +28,7 @@ Key concepts of service-to-pod communication include:
 
 Services use selectors to identify the pods that should receive the traffic. The selectors match labels on the pods to determine which pods are part of the service. Example: A service with the selector `app: myapp` will route traffic to all pods with the label `app: myapp`.
 
-Endpoints are dynamically updated to reflect the current IP addresses of the pods that match the service selector. {product-name} maintains these endpoints and ensures that the service routes traffic to the correct pods.
+Endpoints are dynamically updated to reflect the current IP addresses of the pods that match the service selector. {product-title} maintains these endpoints and ensures that the service routes traffic to the correct pods.
 
 The communication flow refers to the sequence of steps and interactions that occur when a service in Kubernetes routes traffic to the appropriate pods. The typical communication flow for service-to-pod communication is as follows: