net: NAD change ref STP

stps/sig-network/hotpluggable-nad-ref.md

@@ -0,0 +1,301 @@
+# Openshift-virtualization-tests Test plan
+
+## **Live Update NetworkAttachmentDefinition Reference  - Quality Engineering Plan**
+
+### **Metadata & Tracking**
+
+- **Enhancement(s):** https://github.com/kubevirt/enhancements/blob/main/veps/sig-network/hotpluggable-nad-ref.md
+- **Feature Tracking:** https://issues.redhat.com/browse/VIRTSTRAT-560
+- **Epic Tracking:** https://redhat.atlassian.net/browse/CNV-72329
+- **QE Owner(s):** Asia Khromov (@azhivovk)
+- **Owning SIG:** sig-network
+- **Participating SIGs:** sig-network
+
+**Document Conventions (if applicable):**
+- NAD = NetworkAttachmentDefinition, a Multus secondary network definition
+- DNC = Dynamic Networks Controller (Multus dynamic networks controller)
+
+### **Feature Overview**
+
+This feature enables VM owners to change the secondary network of a running VM without rebooting. When the user updates the network reference in the VM configuration, the VM is reconnected to the new network (e.g. a different VLAN) without requiring a restart (but rather seamlessly migrating the VM), preserving guest interface identity.
+
+The supported scope is bridge binding on live-migratable VMs.
+
+---
+
+### **I. Motivation and Requirements Review (QE Review Guidelines)**
+
+#### **1. Requirement & User Story Review Checklist**
+
+- [x] **Review Requirements**
+  - *List the key D/S requirements reviewed:* Change bridge binding NAD reference on a running VM without requiring a restart; the VM should reconnect to the new network seamlessly.
+
+- [x] **Understand Value and Customer Use Cases**
+  - *Describe the feature's value to customers:* VM owners can re-assign new networks (e.g. a different VLAN) to VMs without reboot, avoiding workload impact and preserving guest interface identity (e.g. MAC).
+  - *List the customer use cases identified:*
+    - As a VM owner, I want to re-assign my VM to a different VLAN without rebooting so that my VM connects to the new network without requiring a restart.
+
+- [x] **Testability**
+  - *Note any requirements that are unclear or untestable:* Core flow is testable (see III. Test Scenarios & Traceability).
+
+- [x] **Acceptance Criteria**
+  - *List the acceptance criteria:*
+    - The VM remains running (not rebooted) after changing the secondary network (e.g. VLAN change)
+    - The VM establishes connectivity on the new secondary network (new VLAN)
+    - The VM no longer has connectivity on the previous secondary network (previous VLAN)
+    - The guest interface identity (e.g. MAC address) is preserved
+  - *Note any gaps or missing criteria:* None
+
+- [x] **Non-Functional Requirements (NFRs)**
+  - *List applicable NFRs and their targets:* Documentation update required.
+  - *Note any NFRs not covered and why:*
+    - Performance: no new performance requirements introduced.
+    - Security: no new RBAC or authentication changes.
+    - Monitoring/Observability: no new metrics or alerts required.
+    - Scalability: no new scale requirements.
+    - UI: no UI changes introduced by this feature.
+
+#### **2. Known Limitations**
+
+- **Non-migratable VMs:** Changing the secondary network on non-migratable VMs is not supported; the VM will not reconnect to the new network without a restart.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **Binding support:** Secondary network reassignment is supported for bridge binding only; other binding types are not in scope for this feature.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **Guest network config:** Feature does not change guest-side configuration; guest may need separate reconfiguration for new network (e.g. IP).
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **Single-node clusters (SNO):** Cannot be supported because migration is not possible.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **Migrating between CNI types:** Explicit non-goal.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **Changing network binding/plugin:** Explicit non-goal.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **Seamless network connectivity until action is completed:** Not guaranteed by design.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+- **No limitation of migration retries because of missing Network Attachment Definition:** Explicit non-goal.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+
+#### **3. Technology and Design Review**
+
+- [x] **Developer Handoff/QE Kickoff**
+  - *Key takeaways and concerns:* Meeting where Dev introduced QE through NAD change in VM, migration condition handling, and compatibility with DNC.
+
+- [x] **Technology Challenges**
+  - *List identified challenges:* The feature relies on live migration, which is not supported on all cluster topologies (e.g. single-node clusters), limiting the environments in which this feature can be tested.
+  - *Impact on testing approach:* Tests must run on multi-node clusters where live migration is available.
+
+- [x] **API Extensions**
+  - *List new or modified APIs:* No new APIs introduced by this feature.
+  - *Testing impact:* N/A
+
+- [x] **Test Environment Needs**
+  - *See environment requirements in Section II.3 and testing tools in Section II.3.1*
+
+- [x] **Topology Considerations**
+  - *Describe topology requirements:* No known limitations when considering different topologies.
+  - *Impact on test design:* No specific impact on test design beyond standard live migration requirements.
+
+### **II. Software Test Plan (STP)**
+
+#### **1. Scope of Testing**
+
+**Testing Goals**
+
+- **[P0]** Verify that a running VM with bridge binding can be reconnected to a different secondary network without rebooting, has connectivity on the new network, and is no longer reachable on the previous network.
+- **[P1]** Verify that a running VM connected to a localnet secondary network can be reconnected to a different localnet network without rebooting and has connectivity on the new network.
+- **[P1]** Verify that multiple secondary networks on the same running VM can each be independently updated and remain reachable.
+- **[P2]** Verify that a VM can be successfully live-migrated (user-triggered) after completing a NAD reference update, and maintains connectivity on the new network.
+- **[P2]** Verify that a VM with an updated secondary network reference maintains connectivity on the new network after cluster upgrade.
+
+**Out of Scope (Testing Scope Exclusions)**
+
+None — reviewed and confirmed that all supported product functionality will be tested this cycle.
+
+**Test Limitations**
+
+None — reviewed and confirmed that no test limitations apply for this release.
+
+#### **2. Test Strategy**
+
+**Functional**
+
+- [x] **Functional Testing** — Validates that the feature works according to specified requirements and user stories
+  - *Details:* Validate NAD reference update on running VMs with bridge binding; verify connectivity on new network and absence of connectivity on previous network.
+
+- [x] **Automation Testing** — Confirms test automation plan is in place for CI and regression coverage (all tests are expected to be automated)
+  - *Details:* All new test scenarios will be automated and integrated into the standard CI lane.
+
+- [x] **Regression Testing** — Verifies that new changes do not break existing functionality
+  - *Details:* Existing secondary network and live migration test suites will run as regression to confirm no impact.
+
+**Non-Functional**
+
+- [ ] **Performance Testing** — Validates feature performance meets requirements (latency, throughput, resource usage)
+  - *Details:* Not applicable; no new performance requirements introduced for this feature.
+
+- [ ] **Scale Testing** — Validates feature behavior under increased load and at production-like scale
+  - *Details:* Not applicable; no new scale requirements introduced for this feature.
+
+- [ ] **Security Testing** — Verifies security requirements, RBAC, authentication, authorization
+  - *Details:* Not applicable; no new RBAC or authentication changes introduced.
+
+- [ ] **Usability Testing** — Validates user experience and accessibility requirements
+  - Does the feature require a UI? If so, ensure the UI aligns with the requirements (UI/UX consistency, accessibility)
+  - Does the feature expose CLI commands? If so, validate usability and that needed information is available (e.g. status conditions, clear output)
+  - Does the feature trigger backend operations that should be reported to the admin? If so, validate that the user receives clear feedback about the operation and its outcome (e.g. status conditions, events, or notifications indicating success or failure)
+  - *Details:* Not applicable; no UI or CLI changes introduced by this feature.
+
+- [ ] **Monitoring** — Does the feature require metrics and/or alerts?
+  - *Details:* Not applicable; no new metrics or alerts required.
+
+**Integration & Compatibility**
+
+- [ ] **Compatibility Testing** — Ensures feature works across supported platforms, versions, and configurations
+  - Does the feature maintain backward compatibility with previous API versions and configurations?
+  - *Details:* Not applicable; no new API versions or configurations introduced.
+
+- [x] **Upgrade Testing** — Validates upgrade paths from previous versions, data migration, and configuration preservation
+  - *Details:* Verify that VMs with updated NAD references retain their network configuration and connectivity after cluster upgrade.
+
+- [ ] **Dependencies** — Blocked by deliverables from other components/products. Identify what we need from other teams before we can test.
+  - *Details:* Not applicable; no blocking external dependencies identified.
+
+- [ ] **Cross Integrations** — Does the feature affect other features or require testing by other teams? Identify the impact we cause.
+  - *Details:* Not applicable; no impact on other features or teams identified.
+
+**Infrastructure**
+
+- [ ] **Cloud Testing** — Does the feature require multi-cloud platform testing?
+  - *Details:* Not applicable; feature requires bare-metal cluster with live migration support.
+
+#### **3. Test Environment**
+
+- **Cluster Topology:** 3-master/3-worker cluster (minimum 2 workers), multi-NIC nodes
+
+- **OCP & OpenShift Virtualization Version(s):** OCP 4.22 with OpenShift Virtualization version 4.22
+
+- **CPU Virtualization:** VT-x (Intel) or AMD-V enabled or ARM
+
+- **Compute Resources:** Minimum per worker node: 8 vCPUs, 32GB RAM
+
+- **Special Hardware:** N/A
+
+- **Storage:** ocs-storagecluster-ceph-rbd-virtualization
+
+- **Network:** OVN-Kubernetes, IPv4, IPv6, secondary bridge network, localnet
+
+- **Required Operators:** NMState Operator (for bridge interface configuration)
+
+- **Platform:** Agnostic
+
+- **Special Configurations:** N/A
+
+#### **3.1. Testing Tools & Frameworks**
+
+- **Test Framework:** Standard
+
+- **CI/CD:** N/A
+
+- **Other Tools:** N/A
+
+#### **4. Entry Criteria**
+
+The following conditions must be met before testing can begin:
+
+- [x] Requirements and design documents are **approved and merged**
+- [x] Test environment can be **set up and configured** (see Section II.3 - Test Environment)
+
+#### **5. Risks**
+
+**Timeline/Schedule**
+
+- [x] **Risk:** None identified.
+  - **Mitigation:** N/A
+  - *Estimated impact on schedule:* N/A
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+**Test Coverage**
+
+- [x] **Risk:** Coverage is limited to bridge binding only; other binding types are not tested.
+  - **Mitigation:** Accepted per feature scope - bridge binding is the only supported binding type for this feature.
+  - *Areas with reduced coverage:* Other network binding types.
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+**Test Environment**
+
+- [x] **Risk:** None identified. Cluster topology requirements are documented in Section II.3.
+  - **Mitigation:** N/A
+  - *Missing resources or infrastructure:* None
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+**Untestable Aspects**
+
+- [x] **Risk:** None identified — all components are testable. Connection continuity during the swap is a non-goal by design and is documented in Known Limitations (Section I.2).
+  - **Mitigation:** N/A
+  - *Alternative validation approach:* N/A
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+**Resource Constraints**
+
+- [x] **Risk:** No special resource requirements.
+  - **Mitigation:** N/A
+  - *Current capacity gaps:* None
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+**Dependencies**
+
+- [x] **Risk:** No blocking external dependencies identified.
+  - **Mitigation:** N/A
+  - *Dependent teams or components:* None
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+**Other**
+
+- [x] **Risk:** None identified.
+  - **Mitigation:** N/A
+  - *Sign-off:* Ronen Sde-Or, 04/2026
+
+---
+
+### **III. Test Scenarios & Traceability**
+
+- **[VIRTSTRAT-560]** — As a VM owner, I want to change my VM's VLAN without rebooting so that my workload remains available during network reconfiguration.
+  - *Test Scenario:* [Tier 2] Verify that a running VM with bridge binding (OVN-Kubernetes) remains running after a VLAN change, establishes connectivity on the new VLAN, and is no longer reachable on the previous VLAN.
+  - *Priority:* P0
+
+- **[VIRTSTRAT-560]** — As a VM owner, I want to re-assign my VM connected to a localnet secondary network to a different network without rebooting.
+  - *Test Scenario:* [Tier 2] Verify that a running VM connected to a localnet secondary network can be reconnected to a different localnet network (e.g. different VLAN) without rebooting and has connectivity on the new network.
+  - *Priority:* P1
+
+- **[VIRTSTRAT-560]** — As a VM owner, I want to update multiple secondary networks on the same running VM and have connectivity on all of them.
+  - *Test Scenario:* [Tier 2] Verify that multiple secondary networks on the same running VM can each be independently updated to different NADs and remain reachable.
+  - *Priority:* P1
+
+- **[VIRTSTRAT-560]** — As a VM owner, I want my VM to be live-migratable (user-triggered) after a NAD reference update.
+  - *Test Scenario:* [Tier 2] Verify that a VM can be successfully live-migrated (user-triggered) after completing a secondary network reassignment, and maintains connectivity on the new network.
+  - *Priority:* P2
+
+- **[VIRTSTRAT-560]** — As a VM owner, I want my VM to remain connected to its updated secondary network after a cluster upgrade.
+  - *Test Scenario:* [Tier 2] Pre-upgrade: update the secondary network reference on a running VM and verify connectivity on the new network. Post-upgrade: verify the VM is still connected to the new network and has connectivity.
+  - *Priority:* P2
+
+---
+
+### **IV. Sign-off and Approval**
+
+This Software Test Plan requires approval from the following stakeholders:
+
+* **Reviewers:**
+  - Leading Dev: Ananya Banerjee (@frenzyfriday)
+  - QE Architect: Ruth Netser (@rnetser)
+* **Approvers:**
+  - QE Architect: Ruth Netser (@rnetser)
+  - Principal Developer: Edward Haas (@EdDev)
+  - Product Manager/Owner: Ronen Sde-Or (@ronensdeor)