Add: st40p add seq tracking, interlace auto-detect, and GST redundant support

doc/st40_validation_updates.md

@@ -9,6 +9,10 @@ This note captures all recent ST40/ST40p feature changes and the accompanying va
 - **Frame-info log surfacing:** When tests pass `log_frame_info=True`, the harness dumps the frame-info file and a summary directly into the test log (no artifact download needed).
 - **ST40P test mutation knobs:** The GStreamer TX plugin accepts `tx-test-mode` with helpers for `no-marker`, `seq-gap`, `bad-parity`, and `paced` (optionally with `tx-test-pkt-count` and `tx-test-pacing-ns`). These drive targeted negative/edge-path tests.
 - **Interlaced & split-mode coverage:** RX/TX can be flagged interlaced independently; mismatch fails fast. Split-mode plus marker handling is now covered in tests.
+- **Interlace auto-detect (default):** RX automatically infers progressive vs interlaced cadence from RTP F bits.
+The `interlaced` field in RX ops serves as the initial value before the first F bits arrive. Set `ST40P_RX_FLAG_DISABLE_AUTO_DETECT` (or `rx-disable-auto-detect=true` in GStreamer) to skip auto-detection and use the `interlaced` field as-is (`interlaced=true` forces interlaced mode, `interlaced=false` forces progressive mode).
+- **Auto-detect reset on discontinuity:** A sequence gap (`seq_discont>0`) clears the interlace detection state; RX re-learns cadence from subsequent F bits and continues logging `second_field`/`interlaced` per field.
+- **Warnings when cadence is unknown:** Pipeline RX and the GStreamer RX plugin log auto-detect status on attach.
 - **Integration safety nets:** New noctx integration tests for ST40 interlaced flows, and expanded GStreamer validation tests across single-host and dual-host (VF/VF) paths.
 
 ## Data path (split ANC per RTP)
@@ -44,6 +48,16 @@ flowchart LR
 - **Negative-path observability:** In interlaced mode, a missing marker or sequence gap can occur on just one field. Frame-info shows the gap on that field only, which is why `test_st40i_split_mode_frame_info_logging` and the noctx `st40i_split_seq_gap_reports_loss` exist—to prove per-field accounting is accurate.
 - **Naming clarity:** The `p` in `st40p` refers to the pipeline API, not progressive-only transport. Interlaced support is a first-class, opt-in property (`tx-interlaced` / `rx-interlaced`) and is exercised in both GStreamer and C integration suites.
 
+## Interlace auto-detect: usage and expected signals
+
+- Auto-detect is active by default. To disable it, set `rx-disable-auto-detect=true` in the GStreamer RX pipeline or set `ST40P_RX_FLAG_DISABLE_AUTO_DETECT` in `st40p_rx_ops.flags` (C API), together with `rx-interlaced=true/false` as desired.
+- TX still needs to emit F bits (set `tx_interlaced=true` in GStreamer or `interlaced=true` in TX ops) so RX can learn cadence.
+- Warnings: RX logs a warning if both `rx-interlaced=false` and auto-detect are disabled. Expect a GST_WARNING from `mtl_st40p_rx` and a pipeline warning from `st40p` if cadence is unknown.
+- Frame-info fields: `second_field` (bool) and `interlaced` are populated once F bits are observed (bit1 set for interlaced; `second_field=true` when F==0x3). These are visible in both C API frame_info and GStreamer frame-info dumps.
+- Expect detection log line when auto-detect flips interlaced on RX (`detected interlaced stream (F=0x2/0x3)`); downstream tests assert on these fields rather than relying on caps only.
+- Discontinuity handling: a session-level sequence hole (e.g., `tx-test-mode=seq-gap` or real loss) clears detection state; expect `seq_discont>0` in frame-info and a subsequent “detected interlaced stream” log once new F bits arrive. This proves re-learn after reset.
+- Build verification: RX attach logs should show `rx_ancillary_session_attach(... flags 0x4 ... interlace)` when `rx-disable-auto-detect=true` propagates; if flags stay `0x0`, the session uses auto-detect and logs `auto`.
+
 ## API and plugin changes (developer-facing)
 
 - **TX (GStreamer st40p):**
@@ -58,7 +72,7 @@ flowchart LR
   - Supports `rx_rtp_ring_size` for split-mode buffering.
 - **C API:**
   - `st40_api.h` / `st40_pipeline_api.h` expose split-mode flags (`ST40_TX_FLAG_SPLIT_ANC_BY_PKT` / `ST40P_TX_FLAG_SPLIT_ANC_BY_PKT`) and the extended frame-info fields (`rtp_marker`, `seq_discont`, `seq_lost`).
-  - `st40_tx_ops` / `st40p_tx_ops` accept `st40_tx_test_config` so C API users can inject the same mutation patterns (no-marker, seq-gap, bad-parity, paced) used by the GStreamer tests.
+  - `st40_tx_ops` / `st40p_tx_ops` accept `st40_tx_test_config` so C API users can inject the same mutation patterns (no-marker, seq-gap, bad-parity, paced) used by the GStreamer tests.  **Debug-only**: the library acts on these patterns only when compiled with `MTL_SIMULATE_PACKET_DROPS` (enabled automatically in debug builds); in release builds the hot-path checks are compiled out.
   - Pipeline RX accumulates packets per field and reports discontinuities.
 
 ## Test coverage (what now runs)
@@ -95,6 +109,8 @@ Across all single-host tests, frame-info is dumped and summarized in pytest logs
   - `st40i_split_multi_packet_roundtrip` — sends multiple ANC blocks per field (sizes 8/6/4) in split mode; expects RX accumulation and nonzero frames.
   - `st40i_split_loopback` — interlaced split-mode loopback with custom fps and framebuff; ensures split/interlace coexist.
   - `st40i_split_seq_gap_reports_loss` — crafts a manual RTP sequence gap and validates `seq_discont`/`seq_lost` are reported via the C API frame_info, mirroring the GStreamer gap test at a lower level.
+- [tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp](tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp)
+  - `st40p_rx_auto_detect_interlace` — TX flags interlaced, RX starts progressive (auto-detect is default); asserts `interlaced`/`second_field` populate on RX frame_info.
 
 These integration tests exercise the C pipeline APIs directly (outside the GStreamer harness) to prove split-mode, parity, marker, and sequence reporting work end-to-end without additional context setup.
 
@@ -107,7 +123,8 @@ These integration tests exercise the C pipeline APIs directly (outside the GStre
    - Missing marker: `tx-test-mode=no-marker`
    - Bad parity: `tx-test-mode=bad-parity`
    - Paced burst: `tx-test-mode=paced tx-test-pkt-count=3 tx-test-pacing-ns=200000`
-4. Inspect log: look for `FrameInfo` lines followed by `FrameInfoSummary` to confirm packet counts and seq accounting.
+4. Auto-detect is active by default. The `interlaced` field in RX ops is used as the initial value; leave it `false` if cadence is unknown and auto-detect will override it from F bits. Set `rx-disable-auto-detect=true` (or `ST40P_RX_FLAG_DISABLE_AUTO_DETECT`) to skip auto-detect.
+5. Inspect log: look for `FrameInfo` lines followed by `FrameInfoSummary` to confirm packet counts and seq accounting.
 
 ## Signals to watch
 
@@ -119,7 +136,7 @@ These integration tests exercise the C pipeline APIs directly (outside the GStre
 
 - For split-mode validation, set RX `frame_info_path` (for logging) and a power-of-two `rx_rtp_ring_size`; the plugin rejects non power-of-two values.
 - Frame-info summaries rely on the regular expression in the harness; unexpected formats will fall back to raw dump only.
-- Interlaced mismatch between TX/RX fails the test early (before file compare).
+- Interlaced mismatch between TX/RX is automatically resolved by auto-detect; RX re-learns cadence from F bits.
 
 ## Quick architecture view (tests + logging)
 
@@ -141,4 +158,8 @@ flowchart TD
 - APIs: [include/st40_api.h](include/st40_api.h), [include/st40_pipeline_api.h](include/st40_pipeline_api.h)
 - Plugin: [ecosystem/gstreamer_plugin/gst_mtl_st40p_tx.c](ecosystem/gstreamer_plugin/gst_mtl_st40p_tx.c), [ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.c](ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.c)
 - Pipeline impl: [lib/src/st2110/pipeline/st40_pipeline_tx.c](lib/src/st2110/pipeline/st40_pipeline_tx.c), [lib/src/st2110/pipeline/st40_pipeline_rx.c](lib/src/st2110/pipeline/st40_pipeline_rx.c)
-- Tests: [tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py](tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py), [tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py](tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py), [tests/integration_tests/noctx/testcases/st40i_tests.cpp](tests/integration_tests/noctx/testcases/st40i_tests.cpp)
+- Tests:
+  - [tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py](tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py)
+  - [tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py](tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py)
+  - [tests/integration_tests/noctx/testcases/st40i_tests.cpp](tests/integration_tests/noctx/testcases/st40i_tests.cpp)
+  - [tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp](tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp)

doc/validation_framework.md

@@ -124,6 +124,8 @@ The `tests/` directory contains test implementations organized by scenario type:
     - Progressive and interlaced GStreamer loops (RFC8331, fps/frame buffer sweeps)
     - Split-mode packetized ANC with frame-info logging (sequence discontinuity, packet totals, RTP marker) and ring-size validation
     - Pacing sanity via RTP sender helpers and ramdisk-backed media fixtures (configure `ramdisk.media` in `configs/test_config.yaml`)
+    - Redundant ST40p/ST40i GStreamer ANC cases with per-port seq-gap scheduling (real payloads, lifted packet caps) and frame-info checks for seq discontinuity/loss logging
+    - Interlace auto-detect on RX (enabled by default); frame-info includes `second_field` (bool) and `interlaced` for detected cadence, and detection resets on `seq_discont` before re-learning from subsequent F bits
   - Backend-specific tests (DMA, kernel socket, etc.)
   - Integration tests (FFmpeg, GStreamer)
 

ecosystem/gstreamer_plugin/README.md

@@ -117,6 +117,12 @@ When `enable-dma-offload` is set to `true`, every plugin that supports DMA will
 the `dma-dev` port is bound and exported as described in `doc/dma.md` before enabling the
 flag.
 
+When `allow-down-ports` is set to `true`, MTL will proceed with initialization even if
+one or both NIC links are down. Any session whose `port` or `port-red` maps to a down
+interface will have that port silently removed from its port list at creation time,
+reducing `num_port` by one. A session whose only port is down will fail to create.
+This flag is useful for redundant-path setups where one leg may be absent at startup.
+
 > **Warning:**
 > Generally, the `log-level`, `dev-port`, `dev-ip`, `tx-queues`, and `rx-queues` are used to initialize the MTLlibrary. As the MTL library handle is shared between MTL
 > GStreamer plugins of the same pipeline, you only need to pass them once when specifying the arguments for the firstly initialized pipeline. Nothing happens when you specify them elsewhere;
@@ -418,9 +424,9 @@ The `mtl_st40p_tx` plugin supports all pad capabilities (the data is not checked
 | input-format          | enum     | Encoding of incoming ANC buffers (`raw-udw`, `rfc8331-packed`, `rfc8331`) | enum            | `raw-udw`     |
 | parse-8331-meta       | gboolean | **Deprecated.** Shortcut for `input-format=rfc8331-packed`.         | TRUE/FALSE       | FALSE         |
 | max-combined-udw-size | uint     | Maximum combined size of all user data words to send in one buffer | 0 to (20 * 255)  | 20 * 255      |
-| tx-test-mode          | enum     | Test-only mutations: `no-marker`, `seq-gap`, `bad-parity`, `paced` (see below). | enum      | none          |
-| tx-test-pkt-count     | uint     | Number of packets to emit for `tx-test-mode` patterns (e.g., paced burst). | 0 to G_MAXUINT | 0            |
-| tx-test-pacing-ns     | uint     | Inter-packet spacing in nanoseconds for `tx-test-mode=paced`.      | 0 to G_MAXUINT   | 0             |
+| tx-test-mode          | enum     | **DEBUG build only.** Test-only mutations: `no-marker`, `seq-gap`, `bad-parity`, `paced` (see below). | enum      | none          |
+| tx-test-pkt-count     | uint     | **DEBUG build only.** Number of packets to emit for `tx-test-mode` patterns (e.g., paced burst). | 0 to G_MAXUINT | 0            |
+| tx-test-pacing-ns     | uint     | **DEBUG build only.** Inter-packet spacing in nanoseconds for `tx-test-mode=paced`.      | 0 to G_MAXUINT   | 0             |
 
 > **Note:**
 > - `raw-udw` streams carry a single ANC packet per frame without per-packet metadata; the element uses `tx-did`/`tx-sdid` for the outgoing headers, so keeping `max-combined-udw-size` small avoids oversized frames.
@@ -459,9 +465,10 @@ gst-launch-1.0 filesrc location=$INPUT ! mtl_st40p_tx tx-queues=4 udp-port=40000
 
 This command sets up the transmission pipeline with the specified parameters and sends the ancillary data using the `mtl_st40p_tx` plugin.
 
-#### 5.1.3. ST40P test-mode knobs
+#### 5.1.3. ST40P test-mode knobs (DEBUG builds only)
 
-Test-only RTP/ANC mutation options for validation (not for production):
+Test-only RTP/ANC mutation options for validation (not for production).  
+These properties and the underlying hot-path logic are compiled only when `MTL_SIMULATE_PACKET_DROPS` is defined (enabled automatically in debug builds via `meson -Dbuildtype=debug`). In release builds the properties are absent and the hot-path stays clean.
 
 | Property              | Description                                                                                       |
 |-----------------------|---------------------------------------------------------------------------------------------------|
@@ -492,6 +499,7 @@ The `mtl_st40p_rx` plugin supports all pad capabilities (the data is not checked
 | timeout             | uint    | Timeout in seconds for blocking frame retrieval             | 0 to 300                    | 60            |
 | frame-info-path     | string  | Optional path to append frame info and sequence stats per frame | N/A                      | NULL          |
 | rx-interlaced       | boolean | Whether the incoming ancillary stream is interlaced         | TRUE/FALSE                  | FALSE         |
+| rx-disable-auto-detect | boolean | Skip auto-detection and use the mode set by `rx-interlaced` (`true` = interlaced, `false` = progressive) | TRUE/FALSE | FALSE |
 | output-format       | enum    | Serialization format for received ancillary data            | `raw-udw` / `rfc8331`       | `raw-udw`     |
 
 When `output-format` is set to `rfc8331`, each ancillary packet is serialized with an 8-byte
@@ -502,6 +510,7 @@ its user data words so that downstream elements receive complete RFC8331 payload
 > **Note:** `rtp-ring-size` values are validated at runtime. If the supplied number is not a power of
 > two the element fails to initialize, matching the behavior enforced inside
 > `gst_mtl_st40p_rx_start()`.
+> **Interlace auto-detect:** Enabled by default. The `rx-interlaced` property serves as the initial value before the first RTP F bits are observed. Sequence discontinuities reset detection; frame-info shows `seq_discont` when it happens and `interlaced` re-populates after new F bits arrive.
 
 #### 5.2.2. Example GStreamer Pipeline for Reception
 

ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.c

@@ -128,6 +128,7 @@ enum {
   PROP_ST40P_RX_RTP_RING_SIZE,
   PROP_ST40P_RX_TIMEOUT,
   PROP_ST40P_RX_INTERLACED,
+  PROP_ST40P_RX_DISABLE_AUTO_DETECT,
   PROP_ST40P_RX_OUTPUT_FORMAT,
   PROP_ST40P_RX_FRAME_INFO_PATH,
   PROP_MAX
@@ -346,6 +347,13 @@ static void gst_mtl_st40p_rx_class_init(Gst_Mtl_St40p_RxClass* klass) {
                            "Set to true if ancillary stream is interlaced", FALSE,
                            G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
 
+  g_object_class_install_property(
+      gobject_class, PROP_ST40P_RX_DISABLE_AUTO_DETECT,
+      g_param_spec_boolean(
+          "rx-disable-auto-detect", "Disable interlace auto-detect",
+          "Skip auto-detection and use the interlaced mode set by rx-interlaced", FALSE,
+          G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
+
   g_object_class_install_property(
       gobject_class, PROP_ST40P_RX_OUTPUT_FORMAT,
       g_param_spec_enum("output-format", "Output Format",
@@ -375,6 +383,7 @@ static void gst_mtl_st40p_rx_init(Gst_Mtl_St40p_Rx* src) {
   src->rtp_ring_size = DEFAULT_RTP_RING_SIZE;
   src->timeout_s = 60;
   src->interlaced = FALSE;
+  src->disable_auto_detect = FALSE;
   src->output_format = GST_MTL_ST40P_RX_OUTPUT_FORMAT_RAW_UDW;
   src->frame_info_path = NULL;
   src->frame_info_fp = NULL;
@@ -413,6 +422,9 @@ static void gst_mtl_st40p_rx_set_property(GObject* object, guint prop_id,
     case PROP_ST40P_RX_INTERLACED:
       src->interlaced = g_value_get_boolean(value);
       break;
+    case PROP_ST40P_RX_DISABLE_AUTO_DETECT:
+      src->disable_auto_detect = g_value_get_boolean(value);
+      break;
     case PROP_ST40P_RX_OUTPUT_FORMAT:
       src->output_format = g_value_get_enum(value);
       break;
@@ -453,6 +465,9 @@ static void gst_mtl_st40p_rx_get_property(GObject* object, guint prop_id, GValue
     case PROP_ST40P_RX_INTERLACED:
       g_value_set_boolean(value, src->interlaced);
       break;
+    case PROP_ST40P_RX_DISABLE_AUTO_DETECT:
+      g_value_set_boolean(value, src->disable_auto_detect);
+      break;
     case PROP_ST40P_RX_OUTPUT_FORMAT:
       g_value_set_enum(value, src->output_format);
       break;
@@ -498,6 +513,7 @@ static gboolean gst_mtl_st40p_rx_start(GstBaseSrc* basesrc) {
   GST_DEBUG_OBJECT(src, "RX START: rtp_ring_size=%d", ops_rx.rtp_ring_size);
 
   ops_rx.interlaced = src->interlaced;
+  if (src->disable_auto_detect) ops_rx.flags |= ST40P_RX_FLAG_DISABLE_AUTO_DETECT;
 
   /* Optional frame info logging */
   if (src->frame_info_path && !src->frame_info_fp) {

ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.h

@@ -72,6 +72,7 @@ struct _Gst_Mtl_St40p_Rx {
   guint rtp_ring_size;
   guint timeout_s;
   gboolean interlaced;
+  gboolean disable_auto_detect;
   gint output_format; /* GstMtlSt40pRxOutputFormat enum value */
   gchar* frame_info_path;
   FILE* frame_info_fp;