Keep sustained off-SF traffic out of the background noise estimate

Hardware captures show that a median and lower quartile can follow the interferer while the 12.5th percentile retains the quiet floor. Use that estimate with existing weighting and bounded rise recovery. Wait for settling after every RX re-arm and allow thirty seconds for a complete idle-sample block.

Add captured-sample, low-tail limitation, and real CLI setter regressions. Preserve RXPS intent when parsing an off command with trailing whitespace. All 162 native cases and 24 Python regressions pass.

docs/radio_receive_calibration.md

@@ -2,29 +2,37 @@
 
 Noise calibration collects 64 idle RSSI samples at least 50 ms apart. A quiet
 block takes about 3.2 seconds. Both continuous RX and temporary RXPS calibration
-windows allow up to 10 seconds; a partial block times out without publishing.
+windows allow up to 30 seconds; a partial block times out without publishing.
 Periodic requests coalesce while a block is active. RXPS resumes after a complete
 block, including a held block, or at timeout. Receive and transmit activity are
 never interrupted just to finish calibration.
 
-The block estimate is the median, retaining fractional dBm. A published update
-uses 25% of the previous floor and 75% of the new median, with a -120 dBm lower
-bound. Initial calibration and explicit gain/tuning changes seed a new baseline.
+The block estimate is the 12.5th percentile: the average of the eighth and
+ninth sorted samples, retaining fractional dBm. A published update uses 25% of
+the previous floor and 75% of the new estimate, with a -120 dBm lower bound. Initial calibration and explicit gain/tuning changes seed a new baseline.
 The old published value remains available while collecting. An AGC reset drops
 partial samples while retaining a valid baseline for channel checks.
 
-A median more than 15 dB above the established floor is held twice. The third
+An estimate more than 15 dB above the established floor is held twice. The third
 consecutive complete high block is accepted, allowing recovery after a permanent
 ambient rise. A normal or quieter complete block clears the hold. Partial
 blocks do not advance it. There is no admission threshold based on the old
 floor, so an old low baseline cannot reject every sample from a higher floor.
 
-The median handles minority outliers. If interference occupies most accepted
-samples, the median eventually represents that energy; it cannot prove whether
-the energy is ambient noise or sustained traffic. The rise hold delays that
-change but does not distinguish the two. Busy-channel completion and recovery
-times therefore depend on traffic. Longer sampling also spends more time in
-continuous RX during calibration than the previous rapid 64-sample average.
+Hardware captures with sustained different-SF/BW traffic showed the median and
+lower quartile following the interferer. The 12.5th percentile retained the
+quiet background in those captures. It needs at least nine background samples
+out of 64 to avoid higher-valued traffic entirely. Nearly continuous
+interference or enough low outliers can still mislead it. The rise hold delays
+large changes but cannot identify their source.
+
+RSSI sampling waits 20 ms after every receive re-arm, including CAD and TX
+completion, to avoid unsettled frontend readings. The ten-second trial window
+often failed to complete during heavy traffic with that settling guard; the
+30-second cap allows more idle opportunities. Partial blocks still time out
+without publishing. Longer collection spends more time in continuous RX when
+RXPS is enabled. A quiet minimum is about 3.2 seconds; real completion time
+also depends on packet activity and main-loop scheduling.
 
 CAD has a bounded wait, stops RX duty cycling before scanning, and re-arms the
 configured receive mode on success, busy detection, or error. A completed or

src/helpers/CommonCLI.cpp

@@ -3550,7 +3550,7 @@ void CommonCLI::handleSetCmd(uint32_t sender_timestamp, char* command, char* rep
       // Remember the intent so the timings can auto-retune when SF/BW change.
       _prefs->rx_ps_level = level;
       _prefs->rx_ps_preamble = preamble_overridden ? preamble : 0;   // 0 = auto (derive from SF)
-    } else if (strcmp(value, "off") != 0) {
+    } else if (parsed.mode != RxPowerSavingArgumentMode::Off) {
       // manual <rx_us> <sleep_us> timings are fixed, not level-derived
       // (the named profiles set level_requested and are handled above)
       _prefs->rx_ps_level = 0;

src/helpers/radiolib/NoiseFloorEstimator.h

@@ -3,13 +3,14 @@
 #include <stdint.h>
 #include <math.h>
 
-// PR #2933's spaced median and bounded rise hold, retaining Cascade's
-// fractional dBm and 25% previous / 75% new weighting. No radio or clock I/O.
+// Spaced idle RSSI sampling with a low-percentile estimate and bounded rise hold.
+// Retains Cascade's fractional dBm and 25% previous / 75% new weighting.
+// No radio or clock I/O.
 class NoiseFloorEstimator {
 public:
   static constexpr uint16_t SAMPLE_COUNT = 64;
   static constexpr uint32_t SAMPLE_INTERVAL_MS = 50;
-  static constexpr uint32_t WINDOW_TIMEOUT_MS = 10000;
+  static constexpr uint32_t WINDOW_TIMEOUT_MS = 30000;
   static constexpr uint8_t MAX_HELD_BLOCKS = 3;
   static constexpr int32_t MAX_RISE_CENTI_DB = 1500;
 
@@ -54,18 +55,22 @@ public:
       }
       _samples[j] = key;
     }
-    int32_t median = (int32_t(_samples[SAMPLE_COUNT / 2 - 1])
-        + _samples[SAMPLE_COUNT / 2]) / 2;
-    if (median < -12000) median = -12000;
-    if (previous_valid && median > floor_centi_dbm + MAX_RISE_CENTI_DB) {
+    // Off-SF traffic can dominate the accepted samples. The median (and even
+    // the lower quartile) followed that traffic in hardware captures. The
+    // 12.5th percentile retained the quiet background while rejecting sparse
+    // low outliers. It still requires enough clean, settled samples per block.
+    const uint16_t upper = SAMPLE_COUNT / 8;
+    int32_t candidate = (int32_t(_samples[upper - 1]) + _samples[upper]) / 2;
+    if (candidate < -12000) candidate = -12000;
+    if (previous_valid && candidate > floor_centi_dbm + MAX_RISE_CENTI_DB) {
       if (++_held_blocks < MAX_HELD_BLOCKS) return false;
     }
     _held_blocks = 0;
     if (previous_valid) {
-      const int32_t weighted = floor_centi_dbm + 3 * median;
+      const int32_t weighted = floor_centi_dbm + 3 * candidate;
       floor_centi_dbm = (weighted - 2) / 4;
     } else {
-      floor_centi_dbm = median;
+      floor_centi_dbm = candidate;
     }
     return true;
   }

src/helpers/radiolib/RadioLibWrappers.cpp

@@ -544,6 +544,9 @@ void RadioLibWrapper::startRecv() {
     noInterrupts();
     state = (state & STATE_INT_READY) | STATE_RX;
     interrupts();
+    // RSSI is unsettled after every RX entry, including CAD/TX completion.
+    // Keep the partial block, but wait for the frontend before its next sample.
+    _nf_sample_from = millis() + NF_CALIB_SETTLE_MS;
     _startrx_fails = 0;
     if (_rx_ps_armed) {
       // (re)base the duty-cycle watchdog on the freshly armed cycle

test/README.md

@@ -139,7 +139,7 @@ does not reflect the GoogleTest count -- run the built binary directly
 | `test_persistent_store_format` | `src/helpers/PersistentStoreFormat.h` | contact-page headers and CRCs, dirty-page state, stable slot allocation, and bounded resumable legacy migration across power loss |
 | `test_power_management` | `src/helpers/PowerManagementUtils.h` | median filtering of a brownout outlier and valid-reading requirements for the boot lock |
 | `test_rx_power_saving` | `src/helpers/radiolib/RXPowerSaving.h` | guarded timer/capture bounds across SF5-SF12, bandwidths, TCXO delays and wire preambles; experimental profiles; retuning and strict CLI parsing |
-| `test_noise_floor_estimator` | `src/helpers/radiolib/NoiseFloorEstimator.h` | spaced median, weighting, outliers, persistent rises, timeout/reset semantics and clock rollover |
+| `test_noise_floor_estimator` | `src/helpers/radiolib/NoiseFloorEstimator.h` | spaced low percentile, captured off-SF traffic, weighting, contamination limits, persistent rises, resets and clock rollover |
 | `test_region_names` | `src/helpers/RegionNameUtils.h` | canonical public-region markers while preserving distinct private and differently named regions |
 | `test_datagram_payload_limits` | `src/helpers/DatagramPayloadLimits.h` | encrypted datagram plaintext ceilings, including the anonymous region-reply prefix and worst-case cipher padding |
 | `test_serial_packet_log` | `src/helpers/SerialPacketLog.h` | bounded USB packet logging and dropped-line reporting |

test/test_noise_floor_estimator/test_noise_floor_estimator.cpp

@@ -19,20 +19,22 @@ TEST(NoiseFloorEstimator, RequiresACompleteSpacedBlock) {
   EXPECT_GT(NoiseFloorEstimator::WINDOW_TIMEOUT_MS, 3150U);
 }
 
-TEST(NoiseFloorEstimator, MedianRejectsMinorityOutliersAndKeepsFractionalDbm) {
+TEST(NoiseFloorEstimator, RejectsSparseLowOutliersAndMajorityTraffic) {
   NoiseFloorEstimator n;
   int32_t floor = 0;
   for (unsigned i = 0; i < 64; ++i) {
-    ASSERT_TRUE(n.add(i < 15 ? -30 : i > 48 ? -127 : -103.25f, i * 50));
+    ASSERT_TRUE(n.add(i < 7 ? -127 : i >= 16 ? -30 : -103.25f, i * 50));
   }
   EXPECT_TRUE(n.publish(floor, false));
   EXPECT_EQ(-10325, floor);
 }
 
-TEST(NoiseFloorEstimator, HandlesEvenMedianAndClampsAtMinus120) {
+TEST(NoiseFloorEstimator, InterpolatesPercentileAndClampsAtMinus120) {
   NoiseFloorEstimator n;
   int32_t floor = 0;
-  for (unsigned i = 0; i < 64; ++i) n.add(i < 32 ? -105.5f : -104.5f, i * 50);
+  for (unsigned i = 0; i < 64; ++i) {
+    n.add(i < 7 ? -106 : i == 7 ? -105.5f : i == 8 ? -104.5f : -104, i * 50);
+  }
   ASSERT_TRUE(n.publish(floor, false));
   EXPECT_EQ(-10500, floor);
   n.reset(); fill(n, -127);
@@ -97,15 +99,61 @@ TEST(NoiseFloorEstimator, SpacingSurvivesMillisRolloverAndRejectsInvalidReads) {
   EXPECT_TRUE(n.add(-105, 0x22));
 }
 
-TEST(NoiseFloorEstimator, MajorityContaminationIsAnExplicitLimit) {
+TEST(NoiseFloorEstimator, NearlyContinuousInterferenceIsAnExplicitLimit) {
   NoiseFloorEstimator n;
   int32_t floor = -10500;
   for (unsigned block = 0; block < 3; ++block) {
     n.reset();
-    for (unsigned i = 0; i < 64; ++i) n.add(i < 48 ? -60 : -105, i * 50);
+    for (unsigned i = 0; i < 64; ++i) n.add(i < 60 ? -60 : -105, i * 50);
     EXPECT_EQ(block == 2, n.publish(floor, true));
   }
-  EXPECT_EQ(-7125, floor); // a median cannot distinguish sustained energy from noise
+  EXPECT_EQ(-7125, floor); // too few quiet samples to distinguish traffic from noise
+}
+
+TEST(NoiseFloorEstimator, LowTailContaminationRequiresSettledReceiverSamples) {
+  NoiseFloorEstimator n;
+  int32_t floor = -10500;
+  for (unsigned i = 0; i < 64; ++i) n.add(i < 15 ? -127 : -105, i * 50);
+  ASSERT_TRUE(n.publish(floor, true));
+  EXPECT_EQ(-11625, floor); // the RX wrapper must exclude unsettled frontend readings
+}
+
+TEST(NoiseFloorEstimator, CapturedOffSfTrafficDoesNotBecomeTheBackgroundFloor) {
+  // Mercerwood Heltec V4: matching T1000-E frames plus dense RAK3401 traffic
+  // at another SF/BW. Sorted centi-dBm captures retain the sparse low outliers.
+  const int16_t quiet[64] = {
+    -8300, -8200, -8100, -8100, -7000, -7000, -7000, -7000,
+    -7000, -7000, -7000, -7000, -7000, -7000, -7000, -7000,
+    -7000, -7000, -7000, -7000, -7000, -7000, -6900, -6900,
+    -6900, -6900, -6900, -6900, -6900, -6900, -6900, -6900,
+    -6900, -6900, -6900, -6900, -6900, -6900, -6900, -6900,
+    -6900, -6900, -6900, -6900, -6900, -6900, -6900, -6900,
+    -6900, -6900, -6900, -6900, -6900, -6900, -6900, -6900,
+    -6900, -6900, -6900, -6900, -6900, -6800, -6500, -1100
+  };
+  const int16_t noisy[64] = {
+    -8200, -8200, -7000, -7000, -6900, -6900, -6900, -6900,
+    -6900, -6900, -6900, -6900, -6900, -6400, -4500, -4200,
+    -3600, -3600, -3500, -3400, -3400, -3300, -3200, -3200,
+    -3100, -3000, -3000, -3000, -2900, -2800, -2600, -2600,
+    -2500, -2400, -2400, -2400, -2300, -2300, -2300, -2300,
+    -2300, -2300, -2300, -2200, -2200, -2200, -2200, -2200,
+    -2100, -2100, -2100, -2000, -2000, -2000, -1900, -1900,
+    -1900, -1900, -1800, -1800, -1800, -1800, -1100, -1100
+  };
+  NoiseFloorEstimator n;
+  int32_t floor = 0;
+  for (unsigned i = 0; i < 64; ++i) n.add(quiet[i] / 100.0f, i * 50);
+  ASSERT_TRUE(n.publish(floor, false));
+  EXPECT_GE(floor, -7300);
+  EXPECT_LE(floor, -6800);
+  for (unsigned block = 0; block < 5; ++block) {
+    n.reset();
+    for (unsigned i = 0; i < 64; ++i) n.add(noisy[i] / 100.0f, i * 50);
+    n.publish(floor, true);
+    EXPECT_GE(floor, -7300);
+    EXPECT_LE(floor, -6800); // a median or lower quartile follows the interferer
+  }
 }
 
 int main(int argc, char** argv) {

test/test_radio_receive_contract.py

@@ -149,7 +149,7 @@ int main() {
     assert(w.isChannelActive());
     assert(state == (STATE_RX | STATE_INT_READY)); // preserves the new packet
   }
-  // A quiet ten-second window completes in ~3.2 s and powersaving resumes.
+  // A quiet bounded window completes in ~3.2 s and powersaving resumes.
   for (bool ps : {false, true}) {
     RadioLibWrapper w;
     w._rx_ps_enabled = w._rx_ps_armed = ps;
@@ -164,6 +164,21 @@ int main() {
     assert(w._rx_ps_armed == ps);
     assert(w.reads == 64); // no unnecessary SPI RSSI reads between accepted samples
   }
+  // Each receive re-arm needs fresh frontend settling before sampling RSSI.
+  {
+    RadioLibWrapper w;
+    w.startRecv();
+    for (; now_ms < 120; ++now_ms) w.loop();
+    assert(w.reads == 0);
+    w.loop();
+    assert(w.reads == 1);
+    now_ms = 200;
+    w.startRecv(); // CAD/TX re-arm must settle again, retaining earlier samples
+    for (; now_ms < 220; ++now_ms) w.loop();
+    assert(w.reads == 1 && w._floor_estimator.count() == 1);
+    w.loop();
+    assert(w.reads == 2 && w._floor_estimator.count() == 2);
+  }
   // Repeated scheduled requests must not reset a partial block.
   {
     RadioLibWrapper w;
@@ -177,7 +192,7 @@ int main() {
   {
     RadioLibWrapper w;
     w.packet = true;
-    for (; now_ms <= 10101; ++now_ms) w.loop();
+    for (; now_ms <= NoiseFloorEstimator::WINDOW_TIMEOUT_MS + 101; ++now_ms) w.loop();
     assert(!w._nf_refresh_requested);
     assert(w._noise_floor_centi_dbm == -10500);
     assert(w.reads == 0);
@@ -221,6 +236,76 @@ int main() {
 '''
 
 class RadioReceiveContractTest(unittest.TestCase):
+    def test_real_rxps_setter_preserves_intent_and_rejects_bad_values(self):
+        common = (ROOT / 'src/helpers/CommonCLI.cpp').read_text()
+        setter = method(common, 'if (memcmp(config, "radio.rxps ", 11) == 0)')
+        calculator = method(common, 'bool CommonCLI::calculateRxPowerSavingLevel(')
+        source = r'''
+#include <cassert>
+#include <cstring>
+#include <initializer_list>
+#include <helpers/CLICommandUtils.h>
+#include <helpers/radiolib/RXPowerSaving.h>
+struct NodePrefs {
+  uint8_t rx_powersaving_enabled = 1, rx_ps_level = 6, rx_ps_preamble = 32, sf = 8;
+  uint32_t rx_ps_rx_us = 40000, rx_ps_sleep_us = 32000;
+  float bw = 62.5;
+};
+struct Callbacks {
+  bool accept = true;
+  unsigned calls = 0;
+  bool setRxPowerSaving(bool, uint32_t, uint32_t) { ++calls; return accept; }
+};
+struct CommonCLI {
+  NodePrefs* _prefs;
+  Callbacks* _callbacks;
+  unsigned saves = 0;
+  void savePrefs() { ++saves; }
+  bool calculateRxPowerSavingLevel(uint32_t, uint8_t, float, uint32_t, uint32_t*, uint32_t*);
+  void set(const char* config, char* reply);
+};
+void appendRxPowerSavingAdjustmentNote(char*, const NodePrefs*, uint8_t, float) {}
+@CALCULATOR@
+void CommonCLI::set(const char* config, char* reply) { @SETTER@ }
+int main() {
+  NodePrefs p;
+  Callbacks callbacks;
+  CommonCLI cli{&p, &callbacks};
+  char reply[160] = {};
+  cli.set("radio.rxps level 8 preamble 32", reply);
+  assert(p.rx_powersaving_enabled && p.rx_ps_level == 8 && p.rx_ps_preamble == 32);
+  const uint32_t rx = p.rx_ps_rx_us, sleep = p.rx_ps_sleep_us;
+  cli.set("radio.rxps off \t", reply);
+  assert(!p.rx_powersaving_enabled && p.rx_ps_level == 8 && p.rx_ps_preamble == 32);
+  assert(p.rx_ps_rx_us == rx && p.rx_ps_sleep_us == sleep && cli.saves == 2);
+  for (const char* command : {"radio.rxps level 264 preamble 32",
+      "radio.rxps level 8 preamble 288", "radio.rxps level 4294967296",
+      "radio.rxps 32 1000", "radio.rxps max extra"}) {
+    cli.set(command, reply);
+    assert(cli.saves == 2 && callbacks.calls == 2);
+    assert(!p.rx_powersaving_enabled && p.rx_ps_level == 8 && p.rx_ps_preamble == 32);
+  }
+  cli.set("radio.rxps 12345 67890", reply);
+  assert(p.rx_powersaving_enabled && p.rx_ps_level == 0 && p.rx_ps_preamble == 0);
+  assert(p.rx_ps_rx_us == 12345 && p.rx_ps_sleep_us == 67890 && cli.saves == 3);
+  callbacks.accept = false;
+  cli.set("radio.rxps max", reply);
+  assert(p.rx_ps_level == 0 && p.rx_ps_rx_us == 12345 && cli.saves == 3);
+  callbacks.accept = true;
+  cli.set("radio.rxps max", reply);
+  assert(p.rx_ps_level == 8 && p.rx_ps_preamble == 16 && cli.saves == 4);
+}
+'''
+        with tempfile.TemporaryDirectory(prefix='meshcore-rxps-cli-') as tmp:
+            cpp, binary = Path(tmp) / 'test.cpp', Path(tmp) / 'test'
+            cpp.write_text(source.replace('@CALCULATOR@', calculator).replace('@SETTER@', setter))
+            result = subprocess.run([os.environ.get('CXX', 'c++'), '-std=c++17', '-O1',
+                '-I', str(ROOT / 'test/mocks'), '-I', str(ROOT / 'src'),
+                str(cpp), '-o', str(binary)], capture_output=True, text=True)
+            self.assertEqual(result.returncode, 0, result.stderr)
+            result = subprocess.run([str(binary)], capture_output=True, text=True)
+            self.assertEqual(result.returncode, 0, result.stderr)
+
     def test_production_transitions(self):
         source = (ROOT / 'src/helpers/radiolib/RadioLibWrappers.cpp').read_text()
         names = [('int16_t','performChannelScanWithTimeout'), ('bool','isPacketPendingOrReceiving'),