docs: record V4 idle pass and accepted Morse limitation [skip ci]

2026-10-20 18:38:04 -07:00
1 parent 265b71d77a
commit 784d2f4964
@@ -14,6 +14,34 @@ when its log identifies the physical device, firmware artifact, artifact hash,
 command result, and cold/warm boot outcome. Do not infer success from a tool's
 exit code when the tool has a documented false-success mode.

+## Known-failure follow-up, 2026-10-20/21 UTC
+
+The [follow-up report](hardware_validation_known_failures_1_18_0_1_2026-10-20.md)
+binds builds, host tests and hardware observations to their actual sources.
+It preserves the earlier failures and the accepted fleet results.
+
+- [x] **Current V4 Full logging-on idle/reopen:** shipping source `0b207a1ec`
+  passed case 3: 600.0913 seconds, zero idle writes, 105.3/20.5 ms post-idle
+  reply/reopen, continuous uptime and zero core errors. Watchdog recoveries
+  stayed 2 -> 2 and boot count 0 -> 0. Original settings, strict binary
+  handback, TTY/browser/stream cleanup passed. Complete application readback
+  matched the shipping image; partition table/OTA metadata were unchanged.
+- [x] **Morse receive setting floor:** configured reception now requires at
+  least 15 WPM, with default 25; TX at 1-14 WPM remains available. The operator
+  accepts false letters for now. Earlier quiet/message failures remain recorded;
+  complete-message accuracy and an absolute physical speed cutoff are not
+  inferred from setting validation.
+
+The independent eight-board nRF52 test-bootloader deployment and original-app
+handoffs are complete with documented USB concurrency and RAK secondary-data
+preservation limits. This is separate from final-source Companion qualification.
+The nRF52 testing hold is released; MeshPocket remains excluded.
+
+The current V4 differs from the physical board in the 18 October failure below.
+Its follow-up cannot establish that earlier USB disappearance's cause or erase
+the failed historical measurement. Remaining release gates keep their latest
+documented status.
+
 ## Mercerwood candidate campaign, 2026-10-18 UTC

 The [current hardware campaign](hardware_validation_mercerwood_1_18_0_1_2026-10-18.md)
@@ -0,0 +1,134 @@
+<!-- meshcore-hosted-doc-link:start -->
+<p class="meshcore-hosted-doc-link"><a href="https://mikecarper.github.io/MeshCore/test-results/hardware_validation_known_failures_1_18_0_1_2026-10-20/">View this page on MeshCore Docs</a>.</p>
+<!-- meshcore-hosted-doc-link:end -->
+
+# Known-failure follow-up, 20-21 October 2026 UTC
+
+Status: **V4 USB PASS**; Morse false letters are an accepted limitation.
+Final-source CI remains in progress. This scoped follow-up preserves failed
+measurements and the operator's limited Morse acceptance. It does not grant broader release approval. The current V4 is a
+different physical board from the 18 October campaign.
+
+Final source: `0b207a1ec3a32eb0a8ad6b8643db211de4921cd4`.
+[Safe evidence summary](hardware_validation_known_failures_1_18_0_1_2026-10-20.safe.json)
+records source bindings, evidence digests and explicit pending fields.
+
+## Builds and host verification
+
+| Check | Source | Result |
+| --- | --- | --- |
+| RAK4631 Full | `0b207a1ec` | PASS: 712432/712704 application bytes, 272 bytes free |
+| T096 Full, FEM on | `0b207a1ec` | Shipping build/package PASS |
+| V4.3 Full, FEM off | `0b207a1ec` | Shipping build/package PASS: 2559496 application bytes |
+| Focused Morse host suite | `0b207a1ec` | 111 methods PASS |
+| Targeted USB host suite | `7278eacb9`, included in final source | 45 methods PASS, including 12 HWCDC TX backport suite methods with actual ISR scenarios |
+| Earlier unchanged RadioProfile suite | `fc72943f3` | 17 methods PASS; not rerun for final source |
+
+RAK Full retains all 13 feature capabilities plus `profile.full`; its build
+used INA260 1.5.4 and MLX90614 2.1.7. Builds and host tests do not qualify RF
+behavior or physical USB stability. In
+[CI run 39523154379](https://github.com/mikecarper/MeshCore/actions/runs/39523154379),
+all 11 non-main jobs are reported green; the main host job is **PENDING**.
+A completed workflow pass is not yet claimed.
+
+## V4 application and native USB
+
+The earlier `4773ac4219666d0f5b4c2bff5eb72aa053014662` application passed full
+readback with unchanged partition table and OTA metadata. An initial private
+adapter's ordinary hard reset left native S3 in forced download mode despite
+correct readback. The pinned flasher's masked forced-download clear and
+watchdog reset restored the application. Its adapter guidance was clarified in
+meshfirmware commit [4ce4f44](https://github.com/mikecarper/meshfirmware/commit/4ce4f44);
+the flasher already had the correct sequence.
+
+The second logging-on native-console trial on `4773ac42` passed 600.1086
+seconds with zero idle command writes. The observed idle phase started at
+2026-10-21 00:13:03.667 UTC. First reply/reopen took 124/21.8 ms; uptime advanced
+378 -> 979 -> 981 seconds. Identity, settings restoration, strict binary handback
+and cleanup passed, with zero core errors. However, watchdog recoveries rose
+from 0 to 2; the last reason was client inactivity (`0x08`). This is a transport
+PASS, not a clean watchdog qualification. The earlier browser startup failure
+and its successful restoration remain recorded separately.
+
+Fix `7278eacb9131d613da719fee814fefcdad3adf23`, included in final source,
+requires 100 ms of continuous presence loss before invalidating the valid
+client lease and recognizes the host-backed physical reader. It preserves
+bus-reset, stall and true-unplug recovery. The finding does not conclusively
+attribute both earlier events to that path.
+
+Ordinary shipping `0b207a1ec` was restored successfully at
+2026-10-21 01:21:32 UTC. All 2559496 application bytes matched SHA256
+`04eed20ee0f964fef2a5a0f74f4aafe4b11e0a71b011ff757efc96c03a2cfb14`;
+partition table and OTA metadata stayed unchanged. Runtime source/identity,
+absence of the raw diagnostic marker, strict binary handback and clean core
+state passed. This was an application-only update.
+
+Final logging-on native-console case 3 on this shipping image: **PASS**.
+The idle lasted 600.0913 seconds with zero command writes; post-idle reply and
+reopen took 105.3/20.5 ms. Uptime advanced 172 -> 773 -> 775 seconds, with
+zero core errors. Watchdog recoveries stayed 2 -> 2 and boot count 0 -> 0.
+The watchdog's `boot` field counts reset authorizations, not all physical boots;
+the continuous uptime separately establishes no MCU restart during this case.
+Identity, original preferences, strict binary handback, TTY restoration and
+owned browser/stream cleanup passed. The wrapper finished at
+2026-10-21 01:34:33 UTC without errors. This qualifies the current board and
+artifact for this finite logging-on idle/reopen case; it does not establish
+the cause of the different 18 October board's USB disappearance.
+
+## Morse measurements and accepted limitation
+
+These shipping controls use automatic detection without HIL threshold overrides.
+
+| No intentional TX, 30 seconds | RAK4631 | T096 |
+| --- | --- | --- |
+| Unchanged `08650f64` | FAIL: 3 false characters | FAIL: 1 false character |
+| Adaptive `4773ac42` | PASS: 0 characters in this control | UNQUALIFIED: 6 retained characters and command timeout during external-actor contention |
+
+The contended T096 observation is not a clean detector pass/failure attribution.
+The RAK control alone does not qualify complete messages or other board families.
+
+An unchanged `08650f64` V3 transmitted to a shipping `4773ac42` V4. The quiet
+control came first: **FAIL**, 26 false characters in 300.820761 seconds.
+Three messages at 25/60 WPM, two repeats, then produced **4/12 exact passes and
+8 failures**. Both preferences and strict binary handbacks passed; the wrapper
+reported no errors. This one-way campaign does not replace the original two-way
+24-message matrix.
+
+The operator accepted false letters for now and stopped further detector/gate
+work. That accepts the false-letter limitation; it does not establish complete
+message accuracy or turn the failed measurements into passes. Further Morse
+experiments are not a mandatory gate in this follow-up. The receive setting
+minimum is 15 WPM, default 25; TX at 1-14 WPM remains available. Hand-timing
+tolerance remains, so setting validation is not an absolute physical speed cutoff.
+
+Separate raw diagnostic captures used production RF settings and automatic
+squelch. Capture integrity, both preference restorations and both strict binary
+handbacks passed. These instrumented captures do not qualify shipping accuracy.
+The ordinary shipping image is restored; raw lab details remain private.
+
+## External nRF52 bootloader handback
+
+The operator reported completion of their separate bootloader testing. Its
+deployment record was also read: **COMPLETE_WITH_KNOWN_LIMITATIONS**, eight live
+INFO/original-application handoff proofs and gateway service handback PASS.
+It uses unpublished test bootloader `TEST_0x02040B7E`, source
+`b07255d4cdc601f1313e4cbe020caad3b5544821`, not a latest-stable release.
+The record reports no publication or push by that deployment.
+
+Limits remain: scheduled concurrent USB control failed with one EPIPE and 31
+successful requests; 11 unique bulk halt-clear cycles passed. RAK secondary-data
+preservation is uncertain after earlier manual scratch overlap. Identity and
+seven settings match, without a complete immediately prior contact/channel
+baseline. These records do not qualify final-source `0b207a1ec` applications.
+The nRF52 hold is released; no new nRF52 campaign was started. MeshPocket remains
+excluded from all further testing in this session.
+
+## Remaining scope
+
+Current CI completion and final release artifact-matrix binding remain pending.
+The scoped V4 logging-on hardware case is complete. The [20 October fleet qualification](hardware_validation_mercerwood_1_18_0_1_2026-10-20.md)
+retains its accepted USB/BLE/RF/contact coverage at its actual sources and
+conditions. This report neither reopens those closures nor extends them to
+unmeasured artifacts. Historical failures in the
+[18 October report](hardware_validation_mercerwood_1_18_0_1_2026-10-18.md)
+remain visible. Private device identities, addresses and raw responses are omitted.
@@ -0,0 +1,256 @@
+{
+  "schema": 1,
+  "report_date": "2026-10-20",
+  "campaign_dates_utc": [
+    "2026-10-20",
+    "2026-10-21"
+  ],
+  "status": "V4_USB_PASS_MORSE_LIMITATION_ACCEPTED_CI_PENDING",
+  "release_approval": false,
+  "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+  "scope": {
+    "current_v4_differs_from_18_october_board": true,
+    "nrf52_hold_released_by_operator": true,
+    "new_nrf52_final_source_hardware_qualification_claimed": false,
+    "mesh_pocket_future_testing_excluded": true,
+    "existing_fleet_closures_reopened": false,
+    "device_identifiers_and_host_addresses_omitted": true
+  },
+  "builds": [
+    {
+      "target": "RAK_4631_companion_radio_full",
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "status": "PASS",
+      "application_bytes": 712432,
+      "application_limit_bytes": 712704,
+      "free_bytes": 272,
+      "feature_capability_count": 13,
+      "profile_full_marker": true,
+      "ina260_library_version": "1.5.4",
+      "mlx90614_library_version": "2.1.7",
+      "artifact_filename": "RAK_4631_companion_radio_full-v1.18.0.1-halo-keymind-cascade-dev-0b207a1e.zip",
+      "artifact_sha256": "3126f6eae26a640985bfaeed68fbc2c65f6f99857b326d6d086a35aec8b30f13",
+      "artifact_bytes": 713315
+    },
+    {
+      "target": "Heltec_t096_companion_radio_full_femon",
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "status": "PASS",
+      "artifact_filename": "Heltec_t096_companion_radio_full_femon-v1.18.0.1-halo-keymind-cascade-dev-0b207a1e.zip",
+      "artifact_sha256": "95bcc8c2ba4456a646b21928589c50e17ebd26f8bd866dd01e6862c55ed4d85f",
+      "artifact_bytes": 646782
+    },
+    {
+      "target": "heltec_v4_3_companion_radio_full_femoff",
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "status": "PASS",
+      "application_bytes": 2559496,
+      "artifact_sha256": "04eed20ee0f964fef2a5a0f74f4aafe4b11e0a71b011ff757efc96c03a2cfb14",
+      "instrumentation_hil": false,
+      "artifact_filename": "heltec_v4_3_companion_radio_full_femoff-v1.18.0.1-halo-keymind-cascade-dev-0b207a1e.bin",
+      "artifact_bytes": 2559496
+    }
+  ],
+  "host_checks": {
+    "morse": {
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "status": "PASS",
+      "methods": 111,
+      "elapsed_seconds": 58.472,
+      "log_sha256": "90935b6e37090a425931c98b528d7f56cbd1c9639ca5181db59ec6158e48d27e"
+    },
+    "targeted_usb": {
+      "source_commit": "7278eacb9131d613da719fee814fefcdad3adf23",
+      "included_in_final_source": true,
+      "status": "PASS",
+      "methods": 45,
+      "included_hwcdc_tx_backport_suite_methods": 12,
+      "backport_suite_includes_actual_isr_scenarios": true
+    },
+    "radio_profile": {
+      "source_commit": "fc72943f3bbdcb2d587a39441c82a461d5df68e4",
+      "status": "PASS",
+      "methods": 17,
+      "unchanged_after_run": true,
+      "rerun_for_final_source": false
+    }
+  },
+  "ci": {
+    "run_id": 39523154379,
+    "url": "https://github.com/mikecarper/MeshCore/actions/runs/39523154379",
+    "status": "PENDING",
+    "non_main_jobs_reported_green": 11,
+    "main_host_job": "PENDING",
+    "completed_workflow_pass_claimed": false
+  },
+  "v4_native_console": {
+    "historical_logging_on_case2": {
+      "source_commit": "4773ac4219666d0f5b4c2bff5eb72aa053014662",
+      "status": "TRANSPORT_PASS_WATCHDOG_NOT_QUALIFIED",
+      "observed_idle_started_utc": "2026-10-21T00:13:03.667Z",
+      "idle_seconds": 600.1086,
+      "idle_command_writes": 0,
+      "first_post_idle_reply_ms": 124,
+      "reopened_reply_ms": 21.8,
+      "uptime_seconds": [
+        378,
+        979,
+        981
+      ],
+      "core_error_flags": 0,
+      "watchdog_recoveries_before": 0,
+      "watchdog_recoveries_after": 2,
+      "last_watchdog_reason": "0x08",
+      "last_watchdog_reason_meaning": "client_inactivity",
+      "identity_settings_binary_handback_cleanup": "PASS",
+      "native_safe_evidence_sha256": "8f5f9e9060ab1643575caf121bd1ee0cf703cecf0e02548f5b9c6c7f85701adf",
+      "earlier_browser_startup_failure_preserved": true
+    },
+    "presence_fix": {
+      "source_commit": "7278eacb9131d613da719fee814fefcdad3adf23",
+      "included_in_final_source": true,
+      "continuous_presence_loss_debounce_ms": 100,
+      "host_backed_physical_reader_recognized": true,
+      "bus_reset_stall_and_true_unplug_recovery_preserved": true,
+      "both_historical_events_conclusively_attributed": false
+    },
+    "ordinary_shipping_restore": {
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "status": "PASS",
+      "finished_utc": "2026-10-21T01:21:32.396048Z",
+      "application_only": true,
+      "application_bytes": 2559496,
+      "artifact_and_readback_sha256": "04eed20ee0f964fef2a5a0f74f4aafe4b11e0a71b011ff757efc96c03a2cfb14",
+      "whole_application_readback_match": true,
+      "partition_table_unchanged": true,
+      "otadata_unchanged": true,
+      "runtime_source_identity_and_binary_handback": "PASS",
+      "raw_diagnostic_marker_absent": true,
+      "core_error_flags": 0,
+      "summary_sha256": "ee6a5fa3dbbf8c902e2b2f418831730ef6d91ca42d314dba5b3dd67ef59a1203"
+    },
+    "final_logging_on_case3": {
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "status": "PASS",
+      "required_idle_seconds": 600,
+      "required_unchanged_recovery_and_boot_counters": true,
+      "measured_idle_seconds": 600.0913000000119,
+      "recovery_counters": {
+        "before": 2,
+        "after": 2
+      },
+      "boot_counters": {
+        "before": 0,
+        "after": 0
+      },
+      "identity_uptime_settings_and_binary_handback": "PASS",
+      "safe_evidence_sha256": "75f6bfa7738269d4574f63d6efa6a6795282dd7b6ba6f2694a1eaa5e1f0097ac",
+      "idle_command_writes": 0,
+      "summary_sha256": "f674b83d34a64e5a5ae94411303f42ad7448b3c4a1d31a9723b8086e6e66dc48",
+      "first_post_idle_reply_ms": 105.30000001192093,
+      "reopened_reply_ms": 20.5,
+      "uptime_seconds": [
+        172,
+        773,
+        775
+      ],
+      "core_error_flags": 0,
+      "original_preferences_restored": true,
+      "original_tty_restored": true,
+      "native_streams_and_owned_browser_closed": true,
+      "finished_utc": "2026-10-21T01:34:33.510254+00:00"
+    }
+  },
+  "shipping_morse": {
+    "automatic_detection": true,
+    "hil_threshold_overrides": false,
+    "baseline_quiet_30_seconds": {
+      "source_commit": "08650f644b680a5dfaacb95142e3cef8529da9d2",
+      "status": "FAIL",
+      "rak_false_characters": 3,
+      "t096_false_characters": 1
+    },
+    "adaptive_quiet_30_seconds": {
+      "source_commit": "4773ac4219666d0f5b4c2bff5eb72aa053014662",
+      "rak_status": "PASS_CONTROL_ONLY",
+      "rak_false_characters": 0,
+      "t096_status": "UNQUALIFIED_CONTENDED",
+      "t096_retained_characters": 6,
+      "t096_first_command_timeout": true,
+      "t096_detector_failure_attributed": false
+    },
+    "esp32_one_way_campaign": {
+      "transmitter_source_commit": "08650f644b680a5dfaacb95142e3cef8529da9d2",
+      "receiver_source_commit": "4773ac4219666d0f5b4c2bff5eb72aa053014662",
+      "status": "FAIL",
+      "quiet_control_first": true,
+      "quiet_seconds": 300.82076147598855,
+      "quiet_false_characters": 26,
+      "messages": 3,
+      "rates_wpm": [
+        25,
+        60
+      ],
+      "repeats": 2,
+      "exact_message_passes": 4,
+      "exact_message_failures": 8,
+      "substitutes_for_original_two_way_24_trial_matrix": false,
+      "both_preferences_restored": true,
+      "both_strict_binary_handbacks": "PASS",
+      "wrapper_errors": [],
+      "summary_sha256": "40b5d555c177d81ea11a1fa71e73dafb0e015ee4e25d146a82bf986873bc3405"
+    },
+    "operator_acceptance": {
+      "scope": "current_false_letters_limitation",
+      "accepted": true,
+      "further_detector_gate_work_stopped": true,
+      "mandatory_additional_morse_experiments": false,
+      "complete_message_accuracy_acceptance_or_qualification_inferred": false,
+      "failed_measurements_relabelled": false
+    },
+    "final_source_settings": {
+      "receive_minimum_configured_wpm": 15,
+      "default_wpm": 25,
+      "transmit_1_through_14_wpm_available": true,
+      "hand_timing_tolerance_retained": true,
+      "absolute_physical_speed_cutoff_claimed": false
+    },
+    "separate_raw_diagnostics": {
+      "source_commit": "0b207a1ec3a32eb0a8ad6b8643db211de4921cd4",
+      "instrumentation_hil": true,
+      "production_rf_settings": true,
+      "automatic_squelch": true,
+      "capture_integrity": "PASS",
+      "both_preferences_restored": "PASS",
+      "both_strict_binary_handbacks": "PASS",
+      "ordinary_shipping_image_restored": true,
+      "shipping_accuracy_qualification": false,
+      "further_analysis_stopped_at_operator_request": true,
+      "completed_capture_summary_sha256": "b1d06e6ed98273e29da28601f9a65b273a40d0f295e400f93fc99d12c90c9687"
+    }
+  },
+  "external_nrf52_bootloader_deployment": {
+    "evidence_basis": "operator_report_and_independently_read_deployment_record",
+    "status": "COMPLETE_WITH_KNOWN_LIMITATIONS",
+    "source_commit": "b07255d4cdc601f1313e4cbe020caad3b5544821",
+    "test_bootloader_version": "TEST_0x02040B7E",
+    "latest_stable_release_claimed": false,
+    "live_info_and_original_application_handoffs": 8,
+    "gateway_service_handback": "PASS",
+    "published_or_pushed_by_deployment": false,
+    "scheduled_concurrent_control": {
+      "status": "FAIL",
+      "epipe_errors": 1,
+      "successful_requests": 31
+    },
+    "unique_bulk_halt_clear_cycles_passed": 11,
+    "rak_secondary_data_preservation": "UNCERTAIN_AFTER_EARLIER_MANUAL_SCRATCH_OVERLAP",
+    "rak_identity_and_seven_settings_match": true,
+    "rak_complete_immediately_pre_scratch_contact_channel_baseline_available": false,
+    "final_source_application_qualification_claimed": false,
+    "installed_safe_record_sha256": "297cdfa58fb7d0420694422b680ee25de7d267aa81436911389ea9a766c3110a"
+  },
+  "pending": [
+    "current_ci_completion_and_final_release_artifact_matrix_binding"
+  ]
+}
@@ -23,6 +23,13 @@ Seven active nRF52 boards retain the source
 `88d39510140ce6894cf8e74b93cbd28d7387fa21` bootloader baseline; Companion
 application-only updates transferred no bootloader or SoftDevice data.

+That bootloader baseline describes this campaign's measurements. A subsequent
+independent deployment restored the original applications after installing an
+unpublished test bootloader on eight nRF52 boards. Its remaining USB stress
+and RAK data-preservation limits are recorded in the
+[known-failure follow-up](hardware_validation_known_failures_1_18_0_1_2026-10-20.md).
+This does not repeat or extend the source-bound qualifications below.
+
 | Board | Installed source | Required USB / BLE / RF / contacts | Overall |
 | --- | --- | --- | --- |
 | Heltec V4.3, FEM off | 0865 | USB/BLE/contacts PASS; RF retained ABF | PASS |