Enhancing Packet Loss Handling in FPV Scenarios with Increased FEC

[lida2003]

Currently, I’m facing a situation where FEC (Forward Error Correction) has significantly increased, similar to common FPV scenarios involving trees or obstacles. In such cases, RSSI and SNR alone are insufficient, leading to substantial packet loss. This degradation reduces bandwidth and bitrate, and adjusting the k/n settings might help.

Is it possible to add an additional algorithmic layer to better handle rapid packet loss and increased FEC in FPV scenarios?

[sickgreg]

Dynamic fec, yes i have been working on that recently and will release an update. It works best when there is interference on the channel. If i put my wifi router on the same channel and stream a video to my phone over wifi, i can now fly through it and the fec will change from, say 10/15 to as low as 5/15 (whilst decreasing video bitrate proportionally), and at the same time the link_score gets reduced as well to lower the MCS. It's not perfect

[lida2003]

Dynamic fec, yes i have been working on that recently and will release an update.

Great!

It works best when there is interference on the channel. If i put my wifi router on the same channel and stream a video to my phone over wifi, i can now fly through it and the fec will change from, say 10/15 to as low as 5/15 (whilst decreasing video bitrate proportionally)

I thought this dynamic FEC is quite different from current algothim.

and at the same time the link_score gets reduced as well to lower the MCS. It's not perfect

The current algorithm is suitable for long-range and smooth flying, considering factors like RSSI (RF attenuation) and SNR (signal strength compared to background noise).

For dynamic FEC (or whatever we decide to call it), there are several key FPV characteristics—primarily influenced by obstacles:

• Rapid RSSI Drop: RSSI can decrease dramatically within a short period.
• Packet Loss Impact: Significant packet loss can severely degrade video quality.

To stabilize video quality in these scenarios, immediate action is required (best effort):

1. Boost Transmission Power: Increase power to enhance penetration or improve RSSI.
2. Maintain Bandwidth, Adjust K/N, Reduce Bitrate: Gradually increase the K/N value and decrease the bitrate while keeping the total bandwidth constant.
3. Reduce Bandwidth, Adjust K/N and Bitrate: If the situation worsens, gradually reduce the overall bandwidth, adjust the K/N ratio accordingly, and decrease the bitrate to maintain a stable connection and minimize packet loss.

Once the drone exits the critical situation, it can revert to the long-range, smooth-flying algorithm.

Furthermore, interference from other VTX systems or routers may occur. However, there isn’t much we can do in such cases, aside from increasing RF power or dynamically switching channels—though the latter is likely beyond the scope of this discussion.

There are two or three main algorithms to consider:

1. Current Method (RSSI + SNR): Optimized for long-range and smooth flying.
2. Dynamic FEC/Bitrate/Power Adjustment: Enhances FPV features by adapting to varying conditions.
3. Dynamic Channel Switching: Environments with heavy EMI/EMC interference, such as areas with multiple VTX or router systems. // I don't like this... we have to avoid flying in this kind of environment. Just hit the RTL and land the drone safely.

[lida2003]

Here is an interference sample, where the RF is within range. It will blur the video significantly.

The issue follows a very specific pattern:

a) packet drop on Tx side
b) tx request/keyframe request
c) fec happened

• Zoom in on the earlier part of the image

• tx requested

• SNR/RSSI/related scores

UPDATE: When there are dropped packets in airunit, then we should max power, decrease bitrate to overcome the interference.

• dropped packets

• fec

[lida2003]

Here is a FPV Hostile Scenarios with Interference:

• Video frame will be affected, reduce bitrate, max power should be taken action, When packet loss occurs due to CSMA/CD (Carrier Sense Multiple Access with Collision Detection) EDIT: CSMA/CA（Carrier Sense Multiple Access with Collision Avoidance)
• and trasmision packet will lost due to other issue, then there will no CSMA/CD EDIT: CSMA/CA packet drop, fec will recover some lost packet.

Raw data FYI: https://drive.google.com/drive/folders/1se17dQaQ3KvTzYpvNmTVVSwKZBLIuFZJ?usp=sharing
Dynamic bar graph for better understanding the issue:

• FPV Hostile Scenarios with Interference: Packet Collision and Transmission Packet Loss
• FPV Interference Scenarios: Bitrate/SNR/RSSI/IDR/Drop/FEC/Link

UPDATE: Link Score is lower than v0.58.0.

• FPV Interference Scenarios using adaptive link v0.60.0
• Raw data for FPV Interference Scenarios using adaptive link v0.60.0

[lida2003]

Here is my env: previous set to 149 channel, which will confilict with other ap.

Scanning Wi-Fi Signals on Interface: wlx0c9160035b62 ...
-------------------------------------------
Scanning Wi-Fi 2.4GHz channel ...
Processing: [====================] 100%
Scanning Wi-Fi 5.8GHz channel ...
Processing: [====================] 100%
WiFi List (35 Signals):
Channel 1  Frequency 2.412
Channel 2  Frequency 2.417
Channel 3  Frequency 2.422
Channel 4  Frequency 2.427
Channel 5  Frequency 2.432
Channel 6  Frequency 2.437
Channel 7  Frequency 2.442
Channel 8  Frequency 2.447
Channel 9  Frequency 2.452
Channel 10  Frequency 2.457
Channel 11  Frequency 2.462
Channel 36  Frequency 5.180
Channel 40  Frequency 5.200
Channel 44  Frequency 5.220
Channel 48  Frequency 5.240
Channel 52  Frequency 5.260
Channel 56  Frequency 5.280
Channel 60  Frequency 5.300
Channel 64  Frequency 5.320
Channel 100  Frequency 5.500
Channel 104  Frequency 5.520
Channel 108  Frequency 5.540
Channel 112  Frequency 5.560
Channel 116  Frequency 5.580
Channel 120  Frequency 5.600
Channel 124  Frequency 5.620
Channel 128  Frequency 5.640
Channel 132  Frequency 5.660
Channel 136  Frequency 5.680
Channel 140  Frequency 5.700
Channel 149  Frequency 5.745
Channel 153  Frequency 5.765
Channel 157  Frequency 5.785
Channel 161  Frequency 5.805
Channel 165  Frequency 5.825

So, I have tried 144 channel, which seems to be free, but result is the same(Those trees???obstacle???).

[sickgreg]

I see a lot of xtx (packets dropped by driver). Do you know what causes that? Would that be from collisions in noisy environment? I know even in quiet environment it can be caused by encoder bitrate spikes, which is a real problem for sigmastar. We can reduce some of the encoder problems by disabling image sharpening. As in this file more/imx335_fpv_not_sharp.bin (in main/more repo dir). Copy and point to that in majestic.yaml if you want to try it.

I see high penalty because ground station is detecting fec_rec and lost packets and estimating noise. Also i see the fec change from 10/15 to 10/17, 10/20 etc etc, attempting to help reduce packet loss. Not sure if it is working very well though. Looks like i don't know what i am doing. /config/alink_gs.conf has settings you can try for noise and penalty and fec change. The default settings helped a lot in my limited testing to maintain video when the channel is noisy / busy. The link score gets penalized hard when the channel is noisy though. For me, instead of unflyable constant dropouts, it lowers the bitrate and increases fec and i can still fly. It's not the most elegant solution. A clear channel is the best solution. You may tune the min noise and max noise settings. If you run python3 /usr/local/bin/alink_gs --verbose you can see the noise estimation in real time.

Yeah. I'm not a programmer or an expert! Lucky it works at all. Thanks for your testing. Keep up the good work!

[lida2003]

I see a lot of xtx (packets dropped by driver). Do you know what causes that? Would that be from collisions in noisy environment?

I think it might be two points that need to investigate:

• software
• hardware, which I have asked here: log_interval as a non global setting svpcom/wfb-ng#415 (reply in thread) hope someone with better background can tell more about this.

I know even in quiet environment it can be caused by encoder bitrate spikes, which is a real problem for sigmastar. We can reduce some of the encoder problems by disabling image sharpening. As in this file more/imx335_fpv_not_sharp.bin (in main/more repo dir). Copy and point to that in majestic.yaml if you want to try it.

This is kind of software issue imx335_fpv_not_sharp.bin, I'll try. How about increase driver buffer, any helps?

Also i see the fec change from 10/15 to 10/17, 10/20 etc etc, attempting to help reduce packet loss. Not sure if it is working very well though.

I have add fec back, and will keep monitoring if K/N helps.

A clear channel is the best solution. You may tune the min noise and max noise settings.

Yeah, I have checked and used what's considered a "clear channel," but it seems it doesn't help much. I also noticed that the SNR is not high, though the RSSI is acceptable. Based on the data, packet drops occur intermittently.

There are two types of packet drops I have observed:

1. Packet drop in the air: This can be monitored on the ground station (GS).
   ==> High probability of recovery via FEC.

2. Packet drop in the airunit driver: This is definitely not ideal.
   ==> Medium probability of recovery via FEC, but it becomes difficult if more and more packets are lost.

I think we need to investigate why packets are being lost in the driver. Let's try using imx335_fpv_not_sharp.bin.

The previous packet drops in the driver on the airunit were actually useful, as they let us detect the issue in advance. However, if packet drops are observed on the GS side, there is a timing issue that needs to be addressed.

[lida2003]

I know even in quiet environment it can be caused by encoder bitrate spikes, which is a real problem for sigmastar. We can reduce some of the encoder problems by disabling image sharpening. As in this file more/imx335_fpv_not_sharp.bin (in main/more repo dir). Copy and point to that in majestic.yaml if you want to try it.

This is kind of software issue imx335_fpv_not_sharp.bin, I'll try. How about increase driver buffer, any helps?

• imx335_fpv_not_sharp.bin
• increase from 196608 to 262144 doesn't help

I have a few questions here for new 0.60.1/0.60.0

adaptive-link/alink_gs

Line 154 in b24b353

error_ratio = (5 * lost_packets + adjusted_fec_rec) / (all_packets / num_antennas)

adaptive-link/alink.conf

Line 32 in b24b353

allow_rq_kf_by_tx_d=0

1. why it's 5 times of lost_packets, any info or theroy ?
2. all_packets seems to be from the very beginning, it's kind of average error_ratio. why not check error in a period of short time?
3. why disable allow_rq_kf_by_tx_d, I have found it's really necessary to send a key frame as there are lost of packet loss when driver reported?

With the above changes, I have got this: I still think the penalty is too much, anyway to adjust penalty a little bit low, such as from -350 to -175?

• deduction_exponent = 0.5
  

• deduction_exponent = 0.8
  

UPDATE: I noticed there is no packet loss on gs just fec recoverd packet, but there will be penalty about 100+

[sickgreg]

1. why it's 5 times of lost_packets, any info or theroy ?

i don't really have an explanation other than i thought a dropped packet should be weighted higher than fec recovered. As in, when fec cannot recover the data versus when fec can recover the data. Always open to suggestions.

2. all_packets seems to be from the very beginning, it's kind of average error_ratio. why not check error in a period of short time?

have a look at this json output from wfb

{"type": "rx", "timestamp": 1740713354.5729148, "id": "video rx", "tx_wlan": 0, "packets": {"all": [350, 21802536], "all_bytes": [427768, 26688499722], "dec_err": [0, 3944], "dec_ok": [350, 21798592], "fec_rec": [1, 98139], "lost": [0, 107450], "bad": [0, 0], "out": [123, 7471474], "out_bytes": [145651, 8872942026]}, "session": {"fec_type": "VDM_RS", "fec_k": 8, "fec_n": 12, "epoch": 0}, "rx_ant_stats": [{"ant": 257, "freq": 5805, "mcs": 4, "bw": 20, "pkt_recv": 168, "rssi_min": -84, "rssi_avg": -82, "rssi_max": -81, "snr_min": -4, "snr_avg": 11, "snr_max": 17}, {"ant": 256, "freq": 5805, "mcs": 4, "bw": 20, "pkt_recv": 168, "rssi_min": -91, "rssi_avg": -89, "rssi_max": -87, "snr_min": -11, "snr_avg": 3, "snr_max": 11}, {"ant": 1, "freq": 5805, "mcs": 4, "bw": 20, "pkt_recv": 182, "rssi_min": -78, "rssi_avg": -78, "rssi_max": -78, "snr_min": 16, "snr_avg": 18, "snr_max": 21}, {"ant": 0, "freq": 5805, "mcs": 4, "bw": 20, "pkt_recv": 182, "rssi_min": -88, "rssi_avg": -86, "rssi_max": -86, "snr_min": -11, "snr_avg": 5, "snr_max": 14}]}

As far as i understand it: packets:all first value is correct and 2nd value is total from the start

packets = data.get("packets", {})
    all_packets = packets.get("all", [0])[0]

3. why disable allow_rq_kf_by_tx_d, I have found it's really necessary to send a key frame as there are lost of packet loss when driver reported?

Sorry about that. Probably disabled it for testing purposes and forgot to set it back

[sickgreg]

increase from 196608 to 262144 doesn't help

what is this referring to?

[sickgreg]

With the fec_rec in the osd, the osd is updated every ~1 second. The immediate fec_rec value from the ground station is updated 10 times per second. So we only get a snapshot of this value chosen every 10 messages. I think the ground station should count them every second and send a fec_rec per/sec value each time. What do you think?

[sickgreg]

Check out main repo dir. I've uploaded a pdf file: Adaptive-Link Dynamic FEC and Penalty Tuning Guide. I am hopeful that the features can be improved a lot with tuning. My regular test flying environment is very radio quiet. So we need folks to try tuning it in different environments

[lida2003]

increase from 196608 to 262144 doesn't help

what is this referring to?

That's UDP buffer.

With the fec_rec in the osd, the osd is updated every ~1 second. The immediate fec_rec value from the ground station is updated 10 times per second. So we only get a snapshot of this value chosen every 10 messages.

I didn't see much packet loss on gs. as current code don't have these fec/gs lost data, it's really difficult for debuging: #24
Maybe we can merge this for better understanding of the test env.

Line   22: rssi-27 snr22 fec1 lost0 ants:vrx4
	Line   30: rssi-26 snr22 fec0 lost0 ants:vrx4
	Line   38: rssi-27 snr22 fec0 lost0 ants:vrx4
	Line   46: rssi-27 snr22 fec0 lost0 ants:vrx4
	Line   54: rssi-27 snr22 fec0 lost0 ants:vrx4
	Line   62: rssi-23 snr22 fec0 lost0 ants:vrx4
	Line   70: rssi-22 snr22 fec1 lost0 ants:vrx4
	Line   78: rssi-23 snr21 fec0 lost0 ants:vrx4
	Line   86: rssi-20 snr22 fec0 lost0 ants:vrx4
	Line   94: rssi-19 snr22 fec3 lost0 ants:vrx4
	Line  102: rssi-23 snr22 fec1 lost0 ants:vrx4
	Line  110: rssi-25 snr21 fec1 lost0 ants:vrx4
	Line  118: rssi-26 snr22 fec0 lost0 ants:vrx4
	Line  126: rssi-28 snr22 fec0 lost0 ants:vrx4
	Line  134: rssi-26 snr21 fec1 lost0 ants:vrx4
	Line  142: rssi-43 snr22 fec1 lost0 ants:vrx4
	Line  150: rssi-33 snr21 fec2 lost0 ants:vrx4
	Line  158: rssi-45 snr22 fec0 lost0 ants:vrx4
	Line  166: rssi-49 snr22 fec0 lost0 ants:vrx4
	Line  174: rssi-42 snr21 fec0 lost0 ants:vrx4
	Line  182: rssi-45 snr21 fec0 lost0 ants:vrx4
	Line  190: rssi-40 snr21 fec2 lost0 ants:vrx4
	Line  198: rssi-40 snr21 fec1 lost0 ants:vrx4
	Line  206: rssi-48 snr21 fec1 lost0 ants:vrx4
	Line  214: rssi-48 snr21 fec0 lost0 ants:vrx4
	Line  222: rssi-53 snr21 fec1 lost0 ants:vrx4
	Line  230: rssi-53 snr22 fec2 lost0 ants:vrx4
	Line  238: rssi-52 snr21 fec1 lost0 ants:vrx4
	Line  246: rssi-59 snr21 fec2 lost0 ants:vrx4
	Line  254: rssi-59 snr21 fec2 lost0 ants:vrx4
	Line  262: rssi-56 snr19 fec1 lost0 ants:vrx4
	Line  270: rssi-59 snr21 fec2 lost0 ants:vrx4
	Line  278: rssi-59 snr21 fec2 lost0 ants:vrx4
	Line  286: rssi-56 snr20 fec7 lost0 ants:vrx4
	Line  294: rssi-59 snr21 fec10 lost0 ants:vrx4
	Line  302: rssi-56 snr20 fec4 lost0 ants:vrx4
	Line  310: rssi-57 snr21 fec4 lost0 ants:vrx4
	Line  318: rssi-60 snr20 fec4 lost0 ants:vrx4
	Line  326: rssi-50 snr21 fec0 lost0 ants:vrx4
	Line  334: rssi-46 snr22 fec11 lost0 ants:vrx4
	Line  342: rssi-51 snr22 fec10 lost0 ants:vrx4
	Line  350: rssi-48 snr22 fec9 lost0 ants:vrx4
	Line  358: rssi-42 snr22 fec6 lost0 ants:vrx4
	Line  366: rssi-42 snr22 fec2 lost0 ants:vrx4
	Line  374: rssi-43 snr22 fec6 lost0 ants:vrx4
	Line  382: rssi-44 snr22 fec1 lost0 ants:vrx4
	Line  390: rssi-37 snr21 fec2 lost0 ants:vrx4
	Line  398: rssi-28 snr23 fec2 lost0 ants:vrx4
	Line  406: rssi-28 snr21 fec2 lost0 ants:vrx4
	Line  414: rssi-32 snr21 fec1 lost0 ants:vrx4
	Line  422: rssi-29 snr22 fec1 lost0 ants:vrx4
	Line  430: rssi-31 snr22 fec2 lost0 ants:vrx4
	Line  438: rssi-26 snr22 fec1 lost0 ants:vrx4
	Line  446: rssi-27 snr22 fec3 lost0 ants:vrx4
	Line  454: rssi-24 snr22 fec0 lost0 ants:vrx4
	Line  462: rssi-19 snr23 fec2 lost0 ants:vrx4
	Line  470: rssi-15 snr22 fec2 lost0 ants:vrx4
	Line  478: rssi-14 snr22 fec1 lost0 ants:vrx4
	Line  486: rssi-12 snr23 fec0 lost0 ants:vrx4
	Line  494: rssi4 snr23 fec0 lost0 ants:vrx4
	Line  502: rssi-11 snr22 fec2 lost0 ants:vrx4
	Line  510: rssi-11 snr22 fec2 lost0 ants:vrx4
	Line  518: rssi-19 snr22 fec2 lost0 ants:vrx4
	Line  526: rssi-22 snr22 fec0 lost0 ants:vrx4
	Line  534: rssi-26 snr22 fec1 lost0 ants:vrx4
	Line  542: rssi-26 snr22 fec1 lost0 ants:vrx4
	Line  550: rssi-22 snr21 fec2 lost0 ants:vrx4
	Line  558: rssi-17 snr23 fec1 lost0 ants:vrx4
	Line  566: rssi-12 snr22 fec0 lost0 ants:vrx4
	Line  574: rssi-11 snr22 fec1 lost0 ants:vrx4
	Line  582: rssi-14 snr22 fec3 lost0 ants:vrx4
	Line  590: rssi-14 snr23 fec1 lost0 ants:vrx4
	Line  598: rssi-2 snr22 fec1 lost0 ants:vrx4
	Line  606: rssi-20 snr22 fec1 lost0 ants:vrx4
	Line  614: rssi-21 snr22 fec1 lost0 ants:vrx4
	Line  622: rssi-25 snr22 fec1 lost0 ants:vrx4
	Line  630: rssi-19 snr22 fec1 lost0 ants:vrx4
	Line  638: rssi-19 snr22 fec1 lost0 ants:vrx4
	Line  646: rssi-18 snr22 fec1 lost0 ants:vrx4
	Line  654: rssi-27 snr22 fec0 lost0 ants:vrx4
	Line  662: rssi-28 snr22 fec1 lost0 ants:vrx4
	Line  670: rssi-28 snr22 fec1 lost0 ants:vrx4
	Line  678: rssi-25 snr22 fec1 lost0 ants:vrx4
	Line  686: rssi-27 snr21 fec2 lost0 ants:vrx4
	Line  694: rssi-36 snr22 fec2 lost0 ants:vrx4
	Line  702: rssi-41 snr22 fec2 lost0 ants:vrx4
	Line  710: rssi-43 snr22 fec0 lost0 ants:vrx4
	Line  718: rssi-46 snr22 fec2 lost0 ants:vrx4
	Line  726: rssi-48 snr22 fec1 lost0 ants:vrx4
	Line  734: rssi-51 snr22 fec2 lost0 ants:vrx4
	Line  742: rssi-50 snr21 fec1 lost0 ants:vrx4
	Line  750: rssi-57 snr21 fec1 lost0 ants:vrx4
	Line  758: rssi-61 snr21 fec2 lost0 ants:vrx4
	Line  766: rssi-61 snr21 fec2 lost0 ants:vrx4
	Line  774: rssi-63 snr20 fec1 lost0 ants:vrx4
	Line  782: rssi-62 snr20 fec1 lost0 ants:vrx4
	Line  790: rssi-62 snr20 fec2 lost0 ants:vrx4
	Line  798: rssi-60 snr20 fec0 lost0 ants:vrx4
	Line  806: rssi-64 snr19 fec4 lost0 ants:vrx4
	Line  814: rssi-63 snr19 fec0 lost0 ants:vrx4
	Line  822: rssi-68 snr20 fec1 lost0 ants:vrx4
	Line  830: rssi-57 snr21 fec1 lost0 ants:vrx4
	Line  838: rssi-57 snr21 fec1 lost0 ants:vrx4
	Line  846: rssi-56 snr21 fec2 lost0 ants:vrx4
	Line  854: rssi-56 snr21 fec2 lost0 ants:vrx4
	Line  862: rssi-57 snr21 fec0 lost0 ants:vrx4
	Line  870: rssi-54 snr21 fec1 lost0 ants:vrx4
	Line  878: rssi-58 snr20 fec0 lost0 ants:vrx4
	Line  886: rssi-56 snr21 fec11 lost4 ants:vrx4
	Line  894: rssi-51 snr21 fec2 lost0 ants:vrx4
	Line  902: rssi-52 snr22 fec0 lost0 ants:vrx4
	Line  910: rssi-50 snr22 fec2 lost0 ants:vrx4
	Line  918: rssi-50 snr21 fec2 lost0 ants:vrx4
	Line  926: rssi-50 snr21 fec2 lost0 ants:vrx4
	Line  934: rssi-41 snr22 fec1 lost0 ants:vrx4
	Line  942: rssi-48 snr22 fec0 lost0 ants:vrx4
	Line  950: rssi-45 snr21 fec2 lost0 ants:vrx4
	Line  958: rssi-39 snr21 fec0 lost0 ants:vrx4
	Line  966: rssi-26 snr21 fec2 lost0 ants:vrx4
	Line  974: rssi-29 snr21 fec0 lost0 ants:vrx4
	Line  982: rssi-21 snr21 fec1 lost0 ants:vrx4
	Line  990: rssi-14 snr23 fec2 lost0 ants:vrx4
	Line  998: rssi-13 snr23 fec0 lost0 ants:vrx4
	Line 1006: rssi-13 snr22 fec3 lost0 ants:vrx4
	Line 1014: rssi-12 snr21 fec0 lost0 ants:vrx4
	Line 1022: rssi-10 snr23 fec0 lost0 ants:vrx4
	Line 1030: rssi4 snr23 fec1 lost0 ants:vrx4
	Line 1038: rssi2 snr23 fec2 lost0 ants:vrx4
	Line 1046: rssi6 snr23 fec0 lost0 ants:vrx4
	Line 1054: rssi3 snr23 fec2 lost0 ants:vrx4
	Line 1062: rssi4 snr23 fec1 lost0 ants:vrx4
	Line 1070: rssi1 snr23 fec0 lost0 ants:vrx4

Check out main repo dir. I've uploaded a pdf file: Adaptive-Link Dynamic FEC and Penalty Tuning Guide. I am hopeful that the features can be improved a lot with tuning. My regular test flying environment is very radio quiet. So we need folks to try tuning it in different environments

I have a very hostile interference scenarios, but I don't know much about this figures as my previous post, which has some questions:

1. why it's 5 times of lost_packets, any info or theroy ?

2. all_packets seems to be from the very beginning, it's kind of average error_ratio. why not check error in a period of short time? EDIT: I have figured out what it's all about.

3. why disable allow_rq_kf_by_tx_d, I have found it's really necessary to send a key frame as there are lost of packet loss when driver reported?

If I know more about these parameters, then I can tune much more sucessfully.

Here is a much more reasonable data I think.