Move G3 repeater policy tables and MQTT WebSocket buffers to PSRAM
This commit is contained in:
1 parent
a6d472ff9d
commit
74c03cb30f
15 files changed
+1856
-7
No files matched your search
@@ -391,6 +391,9 @@ jobs:
|
||||
python3 -B test/test_flood_retry_bucket_cli.py -v
|
||||
python3 -B test/test_flood_advert_rtc.py -v
|
||||
python3 -B test/test_flood_advert_iram.py -v
|
||||
python3 -B test/test_flood_advert_psram.py -v
|
||||
python3 -B test/test_repeater_scope_psram.py -v
|
||||
python3 -B test/test_ws_transport_psram.py -v
|
||||
python3 -B test/test_ota_security.py -v
|
||||
python3 -B test/test_companion_offline_delivery.py -v
|
||||
python3 -B test/test_room_receive_admission.py -v
|
||||
|
||||
@@ -886,6 +886,12 @@ send retries only that broker every 30 seconds; successful brokers keep their
|
||||
normal status interval. Each slot has a stable, distinct client ID, including
|
||||
when two slots point at the same broker.
|
||||
|
||||
G3 also places the SDK's two padded WebSocket buffers per initialized MQTT
|
||||
client in PSRAM, saving 2,050 internal bytes per client when PSRAM allocation
|
||||
succeeds. G3 repeaters move their complete scope tables and advert-limit
|
||||
history there too. Capacities, saved formats, task stacks and DMA reserves
|
||||
remain unchanged; safe internal fallback is retained.
|
||||
|
||||
Outbox diagnostics are cached by the network owner, so `get mqtt.stats` does
|
||||
not wait for an SDK mutex held by a network write. `rec:N/write` records a
|
||||
failed-write recovery; `no errors` describes the current connected session,
|
||||
|
||||
@@ -190,6 +190,23 @@ other tasks. Available linked internal heap cannot establish how much
|
||||
DMA-capable heap remains or how it is fragmented after boot. Only runtime
|
||||
observations can establish that.
|
||||
|
||||
G3 repeater builds also place their channel/scope tables and full advert-source
|
||||
history in PSRAM. Their capacities and persisted formats remain unchanged.
|
||||
The RAM check still reserves internal fallback for both complete tables;
|
||||
moving them out of BSS does not relax startup admission or the DMA reserve.
|
||||
If neither heap can safely hold a complete table, startup reports the failed
|
||||
allocation and stops rather than running with reduced filtering protection.
|
||||
Other targets keep their existing storage. `MESH_REPEATER_PSRAM_TABLES=0`
|
||||
restores inline storage for a custom G3 build.
|
||||
|
||||
G3's IDF 4 WebSocket wrapper also prefers PSRAM for its two padded 1,025-byte
|
||||
SDK buffers per initialized MQTT client, including clients using plain MQTT.
|
||||
Five initialized clients move 10,250 bytes of payload from internal RAM.
|
||||
Transport objects, task stacks, DMA buffers and radio packet queues keep their
|
||||
existing placement. The existing extra-byte overflow fix and guarded internal
|
||||
fallback remain in force. These figures describe allocation placement, not a
|
||||
measurement of running free heap or qualification of five simultaneous brokers.
|
||||
|
||||
Maximum-load qualification still needs physical boot and sustained reconnect,
|
||||
publish and configuration-change tests with the selected number of brokers,
|
||||
transport schemes, certificates and enabled UART/browser/OTA features. Record
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
<!-- meshcore-hosted-doc-link:start -->
|
||||
<p class="meshcore-hosted-doc-link"><a href="https://mikecarper.github.io/MeshCore/test-results/g3_psram_storage_2026-10-27/">View this page on MeshCore Docs</a>.</p>
|
||||
<!-- meshcore-hosted-doc-link:end -->
|
||||
|
||||
# G3 repeater PSRAM storage, 2026-10-27
|
||||
|
||||
The G3 Full repeater frees **36,776 bytes of static internal RAM** by moving
|
||||
its channel/scope tables and advert-limit history to PSRAM. The MQTT SDK
|
||||
WebSocket buffers also prefer PSRAM: **2,050 bytes per initialized client**,
|
||||
or 10,250 bytes with five initialized clients. Capacities, saved formats and
|
||||
radio queues remain unchanged.
|
||||
|
||||
These are linked measurements and verified allocation sizes. No G3 was
|
||||
flashed; running heap, RF timing and sustained broker load were not measured.
|
||||
The [machine-readable evidence](g3_psram_storage_2026-10-27.safe.json) records
|
||||
the matched builds, hashes and checks.
|
||||
|
||||
## Matched builds
|
||||
|
||||
Baseline: `88bfaebb5`. Final: the production changes accompanying this report.
|
||||
Both use the same G3 Full recipe, version, Cascade profile and USA preset.
|
||||
|
||||
| Measurement | Baseline | Final |
|
||||
| --- | ---: | ---: |
|
||||
| Internal BSS bytes | 142,088 | 105,312 |
|
||||
| Internal data bytes | 23,980 | 23,980 |
|
||||
| Linked internal heap capacity | 199,976 | 236,752 |
|
||||
| Modeled internal requirement | 197,808 | 234,636 |
|
||||
| Conservative admission margin | 2,168 | 2,116 |
|
||||
| Packaged application bytes | 2,156,504 | 2,155,752 |
|
||||
| OTA application-slot bytes | 6,553,600 | 6,553,600 |
|
||||
|
||||
The model still reserves complete internal fallback for both tables: 18,892
|
||||
bytes for scope storage and 17,936 for advert history, including overhead.
|
||||
That explains why the conservative admission margin does not rise with the
|
||||
normal PSRAM-backed free heap. The 16 KiB DMA reserve and existing startup
|
||||
requirements are unchanged. All 16 capability checks pass, with no declared
|
||||
feature reductions; WiFi/LoRa OTA, MQTT, logging, WebConfig, bridges and host
|
||||
KISS remain compiled in.
|
||||
|
||||
## Allocation behavior
|
||||
|
||||
- G3 repeater defaults select PSRAM for all 255 scope entries, 32 direct-scope
|
||||
names, 255 scope requirements and 128 advert-source histories.
|
||||
- Allocation happens once during construction. Array references preserve
|
||||
addresses, complete-array `sizeof`, file formats and rollback behavior.
|
||||
- The pinned ESP32-S3 SDK initializes PSRAM heaps before global constructors;
|
||||
Arduino's later `psramFound()` flag is not used as a constructor-time gate.
|
||||
- Complete internal fallback is allowed only while preserving the existing
|
||||
DMA reserve. Otherwise startup reports the allocation failure and stops.
|
||||
It never silently reduces filter or advert-protection capacity.
|
||||
- `MESH_REPEATER_PSRAM_TABLES=0` restores inline storage. Other boards retain
|
||||
their existing default storage, including RTC/IRAM paths.
|
||||
- The IDF 4 SDK initializes both WS and WSS transports for each MQTT client,
|
||||
even plain MQTT. Both buffers retain the existing 1,025-byte overflow fix.
|
||||
A failed padded allocation destroys that transport safely instead of
|
||||
returning its original undersized buffer. IDF 5 remains a passthrough.
|
||||
|
||||
## Verification
|
||||
|
||||
- Scope storage: three ASan/UBSan tests cover full capacities, identical 18,885
|
||||
persisted bytes, rollback on short writes/rename failures, lifetime, guarded
|
||||
fallback, fatal allocation failure and platform/SDK gates.
|
||||
- Advert history: four ASan/UBSan tests compare all 128 entries against inline
|
||||
storage, including quota, duplicate, badlist and rollover behavior. Existing
|
||||
RTC (eight), IRAM (seven) and native limiter (33 per backend) tests pass.
|
||||
- WebSocket wrapper: three ASan/UBSan tests exercise 17 scenarios across G3,
|
||||
other PSRAM boards, internal-only boards and IDF 4/5, including the extra
|
||||
byte, cleanup, parent lifetime and fallback boundaries.
|
||||
- RAM policy: all 63 tests pass, including six new fallback regressions.
|
||||
Existing policy capacity, rule layout, moderation and CLI tests pass.
|
||||
- Canonical G3 Full package: compile, internal-RAM admission, dual-OTA fit and
|
||||
capability checks pass. PlatformIO ran one process at a time.
|
||||
|
||||
Reproduce the final build with:
|
||||
|
||||
```sh
|
||||
bash build.sh build-firmware Station_G3_ESP32_repeater --full-exact --firmware-version v1.18.0.2-psram-test --profile cascade --radio-preset usa-cascade-fixed --require-ota
|
||||
```
|
||||
|
||||
Physical acceptance remains a G3 boot, persistence, radio traffic and MQTT
|
||||
reconnect/load run with internal/DMA heap monitoring. The maximum five-broker
|
||||
TLS/WSS/outbox workload remains unqualified by these software checks.
|
||||
@@ -0,0 +1,512 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"baseline_commit": "88bfaebb50cd932bca126675a22be72ee57c0f2d",
|
||||
"target": "Station_G3_ESP32_repeater",
|
||||
"physical_validation_performed": false,
|
||||
"maximum_runtime_load_qualified": false,
|
||||
"command": "bash build.sh build-firmware Station_G3_ESP32_repeater --full-exact --firmware-version v1.18.0.2-psram-test --profile cascade --radio-preset usa-cascade-fixed --require-ota",
|
||||
"net_internal_bss_bytes_freed": 36776,
|
||||
"websocket_payload_bytes_per_initialized_client": 2050,
|
||||
"websocket_payload_bytes_for_five_initialized_clients": 10250,
|
||||
"limitations": [
|
||||
"Linked capacity is not running free heap.",
|
||||
"No G3 hardware flashed or measured.",
|
||||
"No physical RF timing, persistence or broker-soak qualification."
|
||||
],
|
||||
"builds": {
|
||||
"baseline": {
|
||||
"memory": {
|
||||
"schema_version": 1,
|
||||
"target": "Station_G3_ESP32_repeater",
|
||||
"platform": "ESP32_PLATFORM",
|
||||
"mcu": "esp32s3",
|
||||
"elf_sha256": "f434ffa380fd55b77abfa67bf0195cd6af5ae0f5409f91a010c31b28cfe6251a",
|
||||
"passed": true,
|
||||
"available_internal_bytes": 199976,
|
||||
"largest_internal_region_bytes": 167208,
|
||||
"required_heap_bytes": 197808,
|
||||
"required_contiguous_bytes": 19168,
|
||||
"components": {
|
||||
"core_tasks_usb_filesystems": 24576,
|
||||
"wireless_stacks": 49152,
|
||||
"mqtt_startup_allowance": 24576,
|
||||
"udp_bridge_and_socket": 5120,
|
||||
"ota_source_scratch": 8192,
|
||||
"browser_terminal_session": 2048,
|
||||
"neighbor_history": 18448,
|
||||
"host_kiss_codec": 2048,
|
||||
"radio_packet_pool": 10240,
|
||||
"client_table": 9104,
|
||||
"flood_filter_table": 19168,
|
||||
"ota_generic_source_buffers": 3616,
|
||||
"ota_encoder_workspace": 1040,
|
||||
"display_pixels_and_driver": 2048,
|
||||
"screen_objects_and_history": 2048,
|
||||
"allocation_and_transient_margin": 16384
|
||||
},
|
||||
"display": "SH1106Display",
|
||||
"full_companion": false,
|
||||
"mqtt": {
|
||||
"budget_scope": "startup minimum with runtime connection admission",
|
||||
"startup_allowance_bytes": 24576,
|
||||
"startup_active_slot_allowance": 2,
|
||||
"runtime_slot_count": 6,
|
||||
"maximum_active_slots": 5,
|
||||
"maximum_active_slots_without_psram": 2,
|
||||
"worker_stack_bytes": 8192,
|
||||
"client_task_stack_bytes": 6144,
|
||||
"client_task_priority": 5,
|
||||
"client_rx_tx_buffer_bytes": 1792,
|
||||
"maximum_slot_stack_buffer_allowance_bytes": 48384,
|
||||
"cold_start_allocation_allowance_bytes": 16384,
|
||||
"cold_start_dma_admission_free_bytes": 32768,
|
||||
"cold_start_dma_admission_contiguous_bytes": 8192,
|
||||
"additional_broker_cold_start_dma_admission_contiguous_bytes": 20480,
|
||||
"restart_allocation_allowance_bytes": 8192,
|
||||
"restart_dma_admission_free_bytes": 24576,
|
||||
"restart_dma_admission_contiguous_bytes": 8192,
|
||||
"additional_broker_restart_dma_admission_contiguous_bytes": 12288,
|
||||
"dma_admission_free_bytes": 16384,
|
||||
"dma_admission_contiguous_bytes": 4096,
|
||||
"publish_contiguous_bytes": 4096,
|
||||
"publish_contiguous_without_psram_bytes": 8000,
|
||||
"publish_contiguous_with_psram_bytes": 4096,
|
||||
"tls_psram_admission_contiguous_bytes": 16000,
|
||||
"additional_broker_cold_start_tls_psram_admission_contiguous_bytes": 32384,
|
||||
"additional_broker_restart_tls_psram_admission_contiguous_bytes": 24192,
|
||||
"additional_broker_tls_psram_admission_contiguous_bytes": 16000,
|
||||
"tls_internal_admission_free_bytes": 61440,
|
||||
"tls_record_allocation_bytes": 17408,
|
||||
"additional_broker_cold_start_tls_internal_admission_contiguous_bytes": 41792,
|
||||
"additional_broker_restart_tls_internal_admission_contiguous_bytes": 33600,
|
||||
"tls_internal_admission_contiguous_bytes": 17408,
|
||||
"additional_broker_tls_internal_admission_contiguous_bytes": 25408,
|
||||
"maximum_runtime_load_qualified": false,
|
||||
"physical_validation_performed": false,
|
||||
"limitations": "Maximum TLS/WSS/outbox workload is not covered by the startup allowance; runtime free DMA heap and fragmentation must satisfy connection admission; additional brokers reserve publication headroom beside startup allocations."
|
||||
},
|
||||
"regions": [
|
||||
{
|
||||
"start": 1070336488,
|
||||
"end": 1070503696
|
||||
},
|
||||
{
|
||||
"start": 1070530560,
|
||||
"end": 1070563328
|
||||
}
|
||||
],
|
||||
"scope": "Linked capacity before runtime allocation; physical boot/load/soak validation remains required.",
|
||||
"artifact_target": "Station_G3_ESP32_repeater",
|
||||
"pio_environment": "Station_G3_ESP32_repeater_observer_mqtt",
|
||||
"files": {
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.bin": "bf8b4130a79beba18faa95e5f0966d07112c2131c9bcb87915e8b070b387adae",
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb-merged.bin": "defb82a9007799ebb03912536948cc99c3d69f9366bd24898c679c7aa3e8d7c0",
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.capabilities.json": "b2d1879bd5cbed1459cbd56f066af9a2951493263aa782a57fe4610222400e70",
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.meshtext-license.txt": "c37d0a453a2491514bac33ed835c1877f207390f134ae439a97b06b004e236d2"
|
||||
}
|
||||
},
|
||||
"capabilities": {
|
||||
"artifact_target": "Station_G3_ESP32_repeater",
|
||||
"build_profile": "full",
|
||||
"build_recipe": {
|
||||
"schema_version": 1,
|
||||
"sha256": "2db8b799c57e5b9ff224b8fec01bbd1ea1b425188e80d6a9e9b75a2b5658e3ff"
|
||||
},
|
||||
"capabilities": [
|
||||
"bridge.espnow",
|
||||
"bridge.udp",
|
||||
"profile.full",
|
||||
"release.full.same_partition_successor",
|
||||
"radio2.kiss.host",
|
||||
"radio2.kiss.usb",
|
||||
"radio2.kiss.tcp",
|
||||
"ota.update.wifi",
|
||||
"ota.update.lora",
|
||||
"cli.retry_preset",
|
||||
"telemetry.history",
|
||||
"ota.cli",
|
||||
"web.webconfig",
|
||||
"logging.usb.packets",
|
||||
"logging.usb.control",
|
||||
"logging.usb.output",
|
||||
"bridge.udp.configuration",
|
||||
"radio.station_g3.auto_slots"
|
||||
],
|
||||
"ota_update_evidence": "firmware fits both OTA application slots; otadata present",
|
||||
"ota_update_methods": [
|
||||
"wifi",
|
||||
"lora"
|
||||
],
|
||||
"ota_update_verified": true,
|
||||
"platform": "ESP32_PLATFORM",
|
||||
"platformio_env": "Station_G3_ESP32_repeater_observer_mqtt",
|
||||
"reductions": [],
|
||||
"schema_version": 2,
|
||||
"target": "Station_G3_ESP32_repeater",
|
||||
"third_party_notices": [
|
||||
{
|
||||
"component": "espp-meshtext",
|
||||
"file": "Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.meshtext-license.txt",
|
||||
"license": "MIT",
|
||||
"sha256": "c37d0a453a2491514bac33ed835c1877f207390f134ae439a97b06b004e236d2"
|
||||
}
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"capability": "radio2.kiss.host",
|
||||
"evidence": "_ZN4mesh8hostkiss5Codec4feedEh; _ZN4mesh8hostkiss5Codec11queueOutputEhPKhj; _ZN4mesh10Dispatcher22queueForeignRadioFrameEPKhjhjNS_20RadioProfileProtocolE",
|
||||
"present": true,
|
||||
"source": "linked ELF symbols"
|
||||
},
|
||||
{
|
||||
"capability": "radio2.kiss.usb",
|
||||
"evidence": "Error: use set kiss on|off; Error: use set kiss.transport usb|tcp; USB: send KISS FEND to start",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "radio2.kiss.tcp",
|
||||
"evidence": "station TCP 5004",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "bridge.espnow",
|
||||
"evidence": "_ZN12ESPNowBridge5beginEv",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "bridge.udp",
|
||||
"evidence": "_ZN9UDPBridge5beginEv",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "ota.update.wifi",
|
||||
"evidence": "Started: http://%s/update",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "ota.update.lora",
|
||||
"evidence": "image_hash MISMATCH after decode",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "cli.retry_preset",
|
||||
"evidence": "retry.preset",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "telemetry.history",
|
||||
"evidence": "telemetry.volt.i2c",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "ota.cli",
|
||||
"evidence": "OTA: status",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "web.webconfig",
|
||||
"evidence": "start webconfig",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "logging.usb.packets",
|
||||
"evidence": "%s: %s, len=%d (type=%d, route=%s, payload_len=%d)",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "logging.usb.control",
|
||||
"evidence": "OK - USB logging %s (saved)",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "logging.usb.output",
|
||||
"evidence": "OK - logging.output %s (saved)",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "bridge.udp.configuration",
|
||||
"evidence": "udp.enabled",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "radio.station_g3.auto_slots",
|
||||
"evidence": "meshcore.capability.station_g3.auto_slots.v1",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
}
|
||||
],
|
||||
"verified": true
|
||||
},
|
||||
"application_bytes": 2156504,
|
||||
"application_slot_bytes": 6553600,
|
||||
"dram_bss_bytes": 142088,
|
||||
"dram_data_bytes": 23980
|
||||
},
|
||||
"final": {
|
||||
"memory": {
|
||||
"schema_version": 1,
|
||||
"target": "Station_G3_ESP32_repeater",
|
||||
"platform": "ESP32_PLATFORM",
|
||||
"mcu": "esp32s3",
|
||||
"elf_sha256": "96c94f1cafd78cbb9ef8429dbef746225ab0f32aa5b23f130fb42c88ea762fda",
|
||||
"passed": true,
|
||||
"available_internal_bytes": 236752,
|
||||
"largest_internal_region_bytes": 203984,
|
||||
"required_heap_bytes": 234636,
|
||||
"required_contiguous_bytes": 19168,
|
||||
"components": {
|
||||
"core_tasks_usb_filesystems": 24576,
|
||||
"wireless_stacks": 49152,
|
||||
"mqtt_startup_allowance": 24576,
|
||||
"udp_bridge_and_socket": 5120,
|
||||
"ota_source_scratch": 8192,
|
||||
"browser_terminal_session": 2048,
|
||||
"neighbor_history": 18448,
|
||||
"host_kiss_codec": 2048,
|
||||
"radio_packet_pool": 10240,
|
||||
"client_table": 9104,
|
||||
"flood_filter_table": 19168,
|
||||
"repeater_scope_storage_fallback": 18892,
|
||||
"repeater_advert_storage_fallback": 17936,
|
||||
"ota_generic_source_buffers": 3616,
|
||||
"ota_encoder_workspace": 1040,
|
||||
"display_pixels_and_driver": 2048,
|
||||
"screen_objects_and_history": 2048,
|
||||
"allocation_and_transient_margin": 16384
|
||||
},
|
||||
"display": "SH1106Display",
|
||||
"full_companion": false,
|
||||
"mqtt": {
|
||||
"budget_scope": "startup minimum with runtime connection admission",
|
||||
"startup_allowance_bytes": 24576,
|
||||
"startup_active_slot_allowance": 2,
|
||||
"runtime_slot_count": 6,
|
||||
"maximum_active_slots": 5,
|
||||
"maximum_active_slots_without_psram": 2,
|
||||
"worker_stack_bytes": 8192,
|
||||
"client_task_stack_bytes": 6144,
|
||||
"client_task_priority": 5,
|
||||
"client_rx_tx_buffer_bytes": 1792,
|
||||
"maximum_slot_stack_buffer_allowance_bytes": 48384,
|
||||
"cold_start_allocation_allowance_bytes": 16384,
|
||||
"cold_start_dma_admission_free_bytes": 32768,
|
||||
"cold_start_dma_admission_contiguous_bytes": 8192,
|
||||
"additional_broker_cold_start_dma_admission_contiguous_bytes": 20480,
|
||||
"restart_allocation_allowance_bytes": 8192,
|
||||
"restart_dma_admission_free_bytes": 24576,
|
||||
"restart_dma_admission_contiguous_bytes": 8192,
|
||||
"additional_broker_restart_dma_admission_contiguous_bytes": 12288,
|
||||
"dma_admission_free_bytes": 16384,
|
||||
"dma_admission_contiguous_bytes": 4096,
|
||||
"publish_contiguous_bytes": 4096,
|
||||
"publish_contiguous_without_psram_bytes": 8000,
|
||||
"publish_contiguous_with_psram_bytes": 4096,
|
||||
"tls_psram_admission_contiguous_bytes": 16000,
|
||||
"additional_broker_cold_start_tls_psram_admission_contiguous_bytes": 32384,
|
||||
"additional_broker_restart_tls_psram_admission_contiguous_bytes": 24192,
|
||||
"additional_broker_tls_psram_admission_contiguous_bytes": 16000,
|
||||
"tls_internal_admission_free_bytes": 61440,
|
||||
"tls_record_allocation_bytes": 17408,
|
||||
"additional_broker_cold_start_tls_internal_admission_contiguous_bytes": 41792,
|
||||
"additional_broker_restart_tls_internal_admission_contiguous_bytes": 33600,
|
||||
"tls_internal_admission_contiguous_bytes": 17408,
|
||||
"additional_broker_tls_internal_admission_contiguous_bytes": 25408,
|
||||
"maximum_runtime_load_qualified": false,
|
||||
"physical_validation_performed": false,
|
||||
"limitations": "Maximum TLS/WSS/outbox workload is not covered by the startup allowance; runtime free DMA heap and fragmentation must satisfy connection admission; additional brokers reserve publication headroom beside startup allocations."
|
||||
},
|
||||
"regions": [
|
||||
{
|
||||
"start": 1070299712,
|
||||
"end": 1070503696
|
||||
},
|
||||
{
|
||||
"start": 1070530560,
|
||||
"end": 1070563328
|
||||
}
|
||||
],
|
||||
"scope": "Linked capacity before runtime allocation; physical boot/load/soak validation remains required.",
|
||||
"artifact_target": "Station_G3_ESP32_repeater",
|
||||
"pio_environment": "Station_G3_ESP32_repeater_observer_mqtt",
|
||||
"files": {
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.bin": "d3a941a6566d66e76b9315f16f76ec7fbb7d233734dbe5c74ec2a96a3b8344b9",
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb-merged.bin": "b4219d24cac25914b6d8f4b09c31868992c310aca21a5e40d03c4148dd0ef402",
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.capabilities.json": "8efba981bbc219c81fedc78dbf04f4f358d77cf368747c6310ef7bc0a88d6d77",
|
||||
"Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.meshtext-license.txt": "c37d0a453a2491514bac33ed835c1877f207390f134ae439a97b06b004e236d2"
|
||||
}
|
||||
},
|
||||
"capabilities": {
|
||||
"artifact_target": "Station_G3_ESP32_repeater",
|
||||
"build_profile": "full",
|
||||
"build_recipe": {
|
||||
"schema_version": 1,
|
||||
"sha256": "8ffd9cd9ad1dd47a035110d7108ea8a178ad4b9ea7c5552bca41f0d9c33fcef3"
|
||||
},
|
||||
"capabilities": [
|
||||
"bridge.espnow",
|
||||
"bridge.udp",
|
||||
"profile.full",
|
||||
"release.full.same_partition_successor",
|
||||
"radio2.kiss.host",
|
||||
"radio2.kiss.usb",
|
||||
"radio2.kiss.tcp",
|
||||
"ota.update.wifi",
|
||||
"ota.update.lora",
|
||||
"cli.retry_preset",
|
||||
"telemetry.history",
|
||||
"ota.cli",
|
||||
"web.webconfig",
|
||||
"logging.usb.packets",
|
||||
"logging.usb.control",
|
||||
"logging.usb.output",
|
||||
"bridge.udp.configuration",
|
||||
"radio.station_g3.auto_slots"
|
||||
],
|
||||
"ota_update_evidence": "firmware fits both OTA application slots; otadata present",
|
||||
"ota_update_methods": [
|
||||
"wifi",
|
||||
"lora"
|
||||
],
|
||||
"ota_update_verified": true,
|
||||
"platform": "ESP32_PLATFORM",
|
||||
"platformio_env": "Station_G3_ESP32_repeater_observer_mqtt",
|
||||
"reductions": [],
|
||||
"schema_version": 2,
|
||||
"target": "Station_G3_ESP32_repeater",
|
||||
"third_party_notices": [
|
||||
{
|
||||
"component": "espp-meshtext",
|
||||
"file": "Station_G3_ESP32_repeater-full-usb-wifi-ota-v1.18.0.2-psram-test-88bfaebb.meshtext-license.txt",
|
||||
"license": "MIT",
|
||||
"sha256": "c37d0a453a2491514bac33ed835c1877f207390f134ae439a97b06b004e236d2"
|
||||
}
|
||||
],
|
||||
"verification": [
|
||||
{
|
||||
"capability": "radio2.kiss.host",
|
||||
"evidence": "_ZN4mesh8hostkiss5Codec4feedEh; _ZN4mesh8hostkiss5Codec11queueOutputEhPKhj; _ZN4mesh10Dispatcher22queueForeignRadioFrameEPKhjhjNS_20RadioProfileProtocolE",
|
||||
"present": true,
|
||||
"source": "linked ELF symbols"
|
||||
},
|
||||
{
|
||||
"capability": "radio2.kiss.usb",
|
||||
"evidence": "Error: use set kiss on|off; Error: use set kiss.transport usb|tcp; USB: send KISS FEND to start",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "radio2.kiss.tcp",
|
||||
"evidence": "station TCP 5004",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "bridge.espnow",
|
||||
"evidence": "_ZN12ESPNowBridge5beginEv",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "bridge.udp",
|
||||
"evidence": "_ZN9UDPBridge5beginEv",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "ota.update.wifi",
|
||||
"evidence": "Started: http://%s/update",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "ota.update.lora",
|
||||
"evidence": "image_hash MISMATCH after decode",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "cli.retry_preset",
|
||||
"evidence": "retry.preset",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "telemetry.history",
|
||||
"evidence": "telemetry.volt.i2c",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "ota.cli",
|
||||
"evidence": "OTA: status",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "web.webconfig",
|
||||
"evidence": "start webconfig",
|
||||
"present": true,
|
||||
"source": "linked image"
|
||||
},
|
||||
{
|
||||
"capability": "logging.usb.packets",
|
||||
"evidence": "%s: %s, len=%d (type=%d, route=%s, payload_len=%d)",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "logging.usb.control",
|
||||
"evidence": "OK - USB logging %s (saved)",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "logging.usb.output",
|
||||
"evidence": "OK - logging.output %s (saved)",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "bridge.udp.configuration",
|
||||
"evidence": "udp.enabled",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
},
|
||||
{
|
||||
"capability": "radio.station_g3.auto_slots",
|
||||
"evidence": "meshcore.capability.station_g3.auto_slots.v1",
|
||||
"present": true,
|
||||
"source": "packaged application"
|
||||
}
|
||||
],
|
||||
"verified": true
|
||||
},
|
||||
"application_bytes": 2155752,
|
||||
"application_slot_bytes": 6553600,
|
||||
"dram_bss_bytes": 105312,
|
||||
"dram_data_bytes": 23980
|
||||
}
|
||||
},
|
||||
"source_sha256": {
|
||||
"src/helpers/RepeaterPsramStorage.h": "440d6c0e9ea82575ddb34ae7fe17592b86c8e37293fde7480f42a67927441cb7",
|
||||
"src/helpers/FloodAdvertLimiter.h": "08fa87e74a8574f4784cac2a9c4327726fc38a5bfa368e5173c2958e48cfeef7",
|
||||
"src/helpers/ESP32WsTransportFix.cpp": "7d90e2e478aae855f4546e7b9dfcbb9b3d9ad725a13979f099f596cbab89760c",
|
||||
"examples/simple_repeater/MyMesh.h": "0b8aaa704953bd9654840287422010e33193998e7123aaa4e8dd087bec86a335",
|
||||
"examples/simple_repeater/MyMesh.cpp": "f901dbe36021dc16b475785da63caf74889a71c075a19d4bcd1c368df8df0d77",
|
||||
"scripts/check_firmware_ram.py": "0a4016444a95f87e0a6ac82159f8f122ab5609067e16aea525f53ece82f362b3"
|
||||
}
|
||||
}
|
||||
@@ -3707,6 +3707,11 @@ MyMesh::MyMesh(mesh::MainBoard &board, mesh::Radio &radio, mesh::MillisecondCloc
|
||||
: mesh::Mesh(radio, ms, rng, rtc, *createObserverPacketManager(32), tables),
|
||||
region_map(key_store), temp_map(key_store),
|
||||
_cli(board, rtc, sensors, region_map, acl, &_prefs, this),
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
flood_channel_scopes(flood_channel_storage.require("channel scope tables").scopes),
|
||||
flood_channel_direct_scopes(flood_channel_storage.require("channel scope tables").direct),
|
||||
flood_channel_scope_requirements(flood_channel_storage.require("channel scope tables").requirements),
|
||||
#endif
|
||||
telemetry(MAX_PACKET_PAYLOAD - 4),
|
||||
discover_limiter(4, 120), // max 4 every 2 minutes
|
||||
anon_limiter(4, 180) // max 4 every 3 minutes
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
#include <helpers/FloodRuleExtensions.h>
|
||||
#include <helpers/GuestLoginSettings.h>
|
||||
#include <helpers/LoginHistory.h>
|
||||
#include <helpers/RepeaterPsramStorage.h>
|
||||
#if defined(ENABLE_OTA)
|
||||
#include <helpers/ota/OtaContext.h>
|
||||
#endif
|
||||
@@ -558,11 +559,34 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks
|
||||
uint8_t flood_packet_filter_blacklist_count;
|
||||
uint8_t flood_packet_filter_blacklist[FLOOD_PACKET_FILTER_BLACKLIST_MAX]
|
||||
[FLOOD_PACKET_FILTER_PATH_ID_SIZE];
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
// Keep real array references: persistence, resets and rollback use sizeof
|
||||
// these complete tables, and routing retains addresses into them.
|
||||
struct FloodChannelStorage {
|
||||
FloodChannelScopeEntry scopes[FLOOD_CHANNEL_SCOPE_SLOTS];
|
||||
char direct[FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS][FLOOD_PACKET_FILTER_SCOPE_NAME_LEN];
|
||||
FloodChannelScopeRequireEntry requirements[FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS];
|
||||
};
|
||||
static_assert(sizeof(FloodChannelScopeEntry) == 36
|
||||
&& sizeof(FloodChannelScopeRequireEntry) == 34
|
||||
&& sizeof(FloodChannelStorage)
|
||||
<= FLOOD_CHANNEL_SCOPE_SLOTS * 36U
|
||||
+ FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS * 32U
|
||||
+ FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS * 34U + 2U,
|
||||
"Update the repeater PSRAM fallback budget in check_firmware_ram.py");
|
||||
mesh::RepeaterPsramStorage<FloodChannelStorage> flood_channel_storage;
|
||||
FloodChannelScopeEntry (&flood_channel_scopes)[FLOOD_CHANNEL_SCOPE_SLOTS];
|
||||
char (&flood_channel_direct_scopes)[FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS]
|
||||
[FLOOD_PACKET_FILTER_SCOPE_NAME_LEN];
|
||||
FloodChannelScopeRequireEntry
|
||||
(&flood_channel_scope_requirements)[FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS];
|
||||
#else
|
||||
FloodChannelScopeEntry flood_channel_scopes[FLOOD_CHANNEL_SCOPE_SLOTS];
|
||||
char flood_channel_direct_scopes[FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS]
|
||||
[FLOOD_PACKET_FILTER_SCOPE_NAME_LEN];
|
||||
FloodChannelScopeRequireEntry
|
||||
flood_channel_scope_requirements[FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS];
|
||||
#endif
|
||||
#if MESH_ENABLE_FLOOD_GROUP_MODERATION
|
||||
FloodGroupModerationEntry flood_group_moderation[FLOOD_GROUP_MODERATION_SLOTS];
|
||||
#endif
|
||||
|
||||
@@ -354,6 +354,27 @@ def requirements(platform, defines, target):
|
||||
engine = integer(defines, "MESH_ENABLE_FLOOD_RULE_ENGINE", int(platform != "STM32_PLATFORM"))
|
||||
slots = integer(defines, "FLOOD_PACKET_FILTER_SLOTS", 63 if engine else 16)
|
||||
parts["flood_filter_table"] = slots * (304 if engine else 40) + 16
|
||||
psram_tables_default = int(platform == "ESP32_PLATFORM" and qualified_repeater
|
||||
and "STATION_G3_ESP32" in defines
|
||||
and "BOARD_HAS_PSRAM" in defines)
|
||||
if platform == "ESP32_PLATFORM" and integer(
|
||||
defines, "MESH_REPEATER_PSRAM_TABLES", psram_tables_default):
|
||||
# The data normally lives in PSRAM, but all capacities also need
|
||||
# safe internal fallback. Moving it out of BSS cannot erase that
|
||||
# requirement or relax startup admission when PSRAM is missing.
|
||||
scopes = integer(defines, "FLOOD_CHANNEL_SCOPE_SLOTS",
|
||||
31 if "CONFIG_IDF_TARGET_ESP32" in defines else 255)
|
||||
required = integer(defines, "FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS", scopes)
|
||||
direct = integer(defines, "FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS",
|
||||
min(scopes, integer(defines, "MAX_REGION_ENTRIES", 32)))
|
||||
parts["repeater_scope_storage_fallback"] = (
|
||||
scopes * 36 + required * 34 + direct * 32 + 2 + 16)
|
||||
if ("PORTABLE_MQTT_OBSERVER" not in defines
|
||||
and not integer(defines, "FLOOD_ADVERT_IRAM_WORD_STORAGE", 0)
|
||||
and not integer(defines, "FLOOD_ADVERT_RTC_FAST_SLOTS", 0)
|
||||
and not integer(defines, "FLOOD_ADVERT_RTC_SLOW_SLOTS", 0)):
|
||||
parts["repeater_advert_storage_fallback"] = (
|
||||
integer(defines, "FLOOD_ADVERT_SOURCE_SLOTS", 128) * 140 + 16)
|
||||
if "room_server" in target.lower() and integer(
|
||||
defines, "MESH_ENABLE_ROOM_FLOOD_RULE_ENGINE", 0):
|
||||
# FloodRuleEngine keeps all 31 persisted rules in one startup heap
|
||||
@@ -473,6 +494,7 @@ def requirements(platform, defines, target):
|
||||
if "expanded_message_previews" in parts:
|
||||
largest = max(largest, 8192 + parts["expanded_message_previews"])
|
||||
for name in ("client_table", "flood_filter_table", "room_flood_rule_table", "neighbor_history",
|
||||
"repeater_scope_storage_fallback", "repeater_advert_storage_fallback",
|
||||
"solarxiao_voltage_history",
|
||||
"ota_context", "screen_objects_and_history", "ota_encoder_workspace",
|
||||
"ota_generic_source_buffers", "ethernet_or_external_ota_workspaces", "ota_target_name_lookup",
|
||||
|
||||
@@ -40,7 +40,7 @@
|
||||
// the fork moves to Arduino core 3.x.
|
||||
//
|
||||
// Scope: this stops the heap corruption and NOTHING else. It does not make an
|
||||
// oversized response fail the handshake — v4.4's read loop also exits on
|
||||
// oversized response fail the handshake; v4.4's read loop also exits on
|
||||
// `header_len < WS_BUFFER_SIZE` going false, and then still accepts the
|
||||
// connection if "Sec-WebSocket-Accept:" was inside those first bytes (it comes
|
||||
// early, so it usually is). ws_connect() therefore returns success on a partial
|
||||
@@ -67,6 +67,11 @@
|
||||
#include "esp_transport.h"
|
||||
#include "esp_transport_ws.h"
|
||||
|
||||
#if defined(STATION_G3_ESP32) && defined(BOARD_HAS_PSRAM)
|
||||
#include "esp_heap_caps.h"
|
||||
#include "MQTTConnectionAdmission.h"
|
||||
#endif
|
||||
|
||||
#ifndef CONFIG_WS_BUFFER_SIZE
|
||||
#define CONFIG_WS_BUFFER_SIZE 1024
|
||||
#endif
|
||||
@@ -103,13 +108,30 @@ esp_transport_handle_t __wrap_esp_transport_ws_init(esp_transport_handle_t paren
|
||||
if (ws != nullptr && ws->buffer != nullptr) {
|
||||
// The buffer is untouched at this point (allocated moments ago inside
|
||||
// __real_esp_transport_ws_init), so a swap is safe.
|
||||
char* padded = (char*)malloc(CONFIG_WS_BUFFER_SIZE + 1);
|
||||
if (padded != nullptr) {
|
||||
free(ws->buffer);
|
||||
ws->buffer = padded;
|
||||
const size_t bytes = CONFIG_WS_BUFFER_SIZE + 1;
|
||||
#if defined(STATION_G3_ESP32) && defined(BOARD_HAS_PSRAM)
|
||||
// Upgrade request/response text is CPU-only. Retain this padded buffer
|
||||
// until the SDK destroys the transport; its ordinary free accepts PSRAM.
|
||||
char* padded = (char*)heap_caps_malloc(bytes, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
|
||||
if (padded == nullptr) {
|
||||
const uint32_t caps = MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA;
|
||||
if (MQTTConnectionAdmission::canFallback(bytes,
|
||||
heap_caps_get_free_size(caps), heap_caps_get_largest_free_block(caps))) {
|
||||
padded = (char*)heap_caps_malloc(bytes, caps | MALLOC_CAP_8BIT);
|
||||
}
|
||||
}
|
||||
// On alloc failure keep the original buffer: same behavior as before
|
||||
// this fix, which is still strictly better than failing init here.
|
||||
#else
|
||||
char* padded = (char*)malloc(bytes);
|
||||
#endif
|
||||
if (padded == nullptr) {
|
||||
// Returning the original 1024-byte buffer would reintroduce the
|
||||
// overflow. This transport has not entered the client's list yet;
|
||||
// destroy releases its context/buffer without destroying its parent.
|
||||
esp_transport_destroy(t);
|
||||
return nullptr;
|
||||
}
|
||||
free(ws->buffer);
|
||||
ws->buffer = padded;
|
||||
}
|
||||
}
|
||||
return t;
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
#include <string.h>
|
||||
#include "FloodAdvertRtcPolicy.h"
|
||||
#include "FloodAdvertIramPolicy.h"
|
||||
#include "RepeaterPsramStorage.h"
|
||||
|
||||
#ifndef FLOOD_ADVERT_SOURCE_SLOTS
|
||||
#if defined(STM32_PLATFORM)
|
||||
@@ -89,6 +90,14 @@ protected:
|
||||
}
|
||||
#else
|
||||
FloodAdvertLimiter(Entry* entries, size_t capacity) : _entries(entries), _capacity(capacity) {}
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
// The derived storage owner is initialized after this base. Bind its full
|
||||
// table before reset() or any limiter operation can inspect the entries.
|
||||
void bindStorage(Entry* entries) {
|
||||
if (!entries || reinterpret_cast<uintptr_t>(entries) % alignof(Entry)) abort();
|
||||
_entries = entries;
|
||||
}
|
||||
#endif
|
||||
#endif
|
||||
FloodAdvertLimiter(const FloodAdvertLimiter&) = delete;
|
||||
FloodAdvertLimiter& operator=(const FloodAdvertLimiter&) = delete;
|
||||
@@ -196,6 +205,19 @@ public:
|
||||
public:
|
||||
StaticFloodAdvertLimiter(Entry* fast, Entry* slow)
|
||||
: FloodAdvertLimiter(_storage, Capacity, fast, slow) { reset(); }
|
||||
#elif MESH_REPEATER_PSRAM_TABLES
|
||||
static_assert(sizeof(Entry) == 140,
|
||||
"Update the advert PSRAM fallback budget in check_firmware_ram.py");
|
||||
struct Storage { Entry entries[Capacity]; };
|
||||
RepeaterPsramStorage<Storage> _storage;
|
||||
public:
|
||||
StaticFloodAdvertLimiter() : FloodAdvertLimiter(nullptr, Capacity) {
|
||||
// Exhaustion must stop startup rather than disabling advert protection
|
||||
// or reducing its source/hash history. No full internal array is retained.
|
||||
if (!_storage.begin("advert source history")) abort();
|
||||
bindStorage(_storage.data()->entries);
|
||||
reset();
|
||||
}
|
||||
#else
|
||||
Entry _storage[Capacity];
|
||||
public:
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
#pragma once
|
||||
|
||||
#ifndef MESH_REPEATER_PSRAM_TABLES
|
||||
#if defined(STATION_G3_ESP32) && defined(ESP32_PLATFORM) && defined(BOARD_HAS_PSRAM) \
|
||||
&& defined(MESH_CLIENT_REPEATER_ONLY) && MESH_CLIENT_REPEATER_ONLY
|
||||
#define MESH_REPEATER_PSRAM_TABLES 1
|
||||
#else
|
||||
#define MESH_REPEATER_PSRAM_TABLES 0
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#if MESH_REPEATER_PSRAM_TABLES != 0 && MESH_REPEATER_PSRAM_TABLES != 1
|
||||
#error "MESH_REPEATER_PSRAM_TABLES must be 0 or 1"
|
||||
#endif
|
||||
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
#if !defined(ESP32_PLATFORM) || !defined(BOARD_HAS_PSRAM)
|
||||
#error "Repeater PSRAM tables require an ESP32 target with PSRAM support"
|
||||
#endif
|
||||
|
||||
#include <sdkconfig.h>
|
||||
#if !defined(CONFIG_IDF_TARGET_ESP32S3) || !CONFIG_IDF_TARGET_ESP32S3 \
|
||||
|| !defined(CONFIG_SPIRAM_BOOT_INIT) || !CONFIG_SPIRAM_BOOT_INIT \
|
||||
|| !defined(CONFIG_SPIRAM_USE_MALLOC) || !CONFIG_SPIRAM_USE_MALLOC
|
||||
#error "Repeater PSRAM tables require ESP32-S3 with boot-initialized PSRAM heaps"
|
||||
#endif
|
||||
|
||||
#include <esp_heap_caps.h>
|
||||
#include <helpers/MQTTConnectionAdmission.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <type_traits>
|
||||
|
||||
namespace mesh {
|
||||
|
||||
// Own only CPU data here. ISR state, locks, task stacks and DMA buffers retain
|
||||
// their existing internal storage. G3's pinned SDK initializes PSRAM-backed
|
||||
// heaps before global constructors, so tables can keep their startup lifetime.
|
||||
template<typename T> class RepeaterPsramStorage {
|
||||
static_assert(std::is_trivial<T>::value,
|
||||
"Repeater PSRAM tables require zero-initialized trivial data");
|
||||
T* value_ = nullptr;
|
||||
public:
|
||||
RepeaterPsramStorage() = default;
|
||||
~RepeaterPsramStorage() { if (value_) heap_caps_free(value_); }
|
||||
RepeaterPsramStorage(const RepeaterPsramStorage&) = delete;
|
||||
RepeaterPsramStorage& operator=(const RepeaterPsramStorage&) = delete;
|
||||
|
||||
bool begin(const char* label = "repeater table") {
|
||||
if (value_) return true;
|
||||
value_ = static_cast<T*>(heap_caps_calloc(
|
||||
1, sizeof(T), MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT));
|
||||
if (!value_) {
|
||||
const uint32_t caps = MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA;
|
||||
if (MQTTConnectionAdmission::canFallback(sizeof(T),
|
||||
heap_caps_get_free_size(caps), heap_caps_get_largest_free_block(caps))) {
|
||||
value_ = static_cast<T*>(heap_caps_calloc(
|
||||
1, sizeof(T), MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT));
|
||||
}
|
||||
}
|
||||
if (!value_) {
|
||||
fprintf(stderr, "Cannot allocate %s (%u bytes); full capacity is required\n",
|
||||
label, static_cast<unsigned>(sizeof(T)));
|
||||
}
|
||||
return value_ != nullptr;
|
||||
}
|
||||
|
||||
T* data() { return value_; }
|
||||
const T* data() const { return value_; }
|
||||
T& require(const char* label) {
|
||||
// Used only while binding the owner's references during construction.
|
||||
// Allocation never moves after pointers escape to routing or persistence.
|
||||
if (!begin(label)) abort();
|
||||
return *value_;
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace mesh
|
||||
#endif
|
||||
@@ -453,6 +453,149 @@ int main() {
|
||||
sensor = ram.requirements("NRF52_PLATFORM", {}, "t114_sensor")
|
||||
self.assertEqual(sensor["components"]["client_table"], 32 * 320 + 16)
|
||||
|
||||
def test_g3_psram_tables_reserve_full_internal_fallback_without_cutting_other_owners(self):
|
||||
flags = {"STATION_G3_ESP32": 1, "BOARD_HAS_PSRAM": 1,
|
||||
"MESH_CLIENT_REPEATER_ONLY": 1, "WITH_MQTT_BRIDGE": 1,
|
||||
"DISPLAY_CLASS": "SH1106Display", "ADMIN_PASSWORD": "test",
|
||||
"ENABLE_OTA": 1, "WITH_UDP_BRIDGE": 1, "WITH_RS232_BRIDGE": "Serial2"}
|
||||
target = "Station_G3_ESP32_repeater"
|
||||
original = ram.requirements("ESP32_PLATFORM", {
|
||||
**flags, "MESH_REPEATER_PSRAM_TABLES": 0}, target)
|
||||
selected = ram.requirements("ESP32_PLATFORM", flags, target)
|
||||
scope = 255 * 36 + 255 * 34 + 32 * 32 + 2 + 16
|
||||
advert = 128 * 140 + 16
|
||||
self.assertEqual(selected["components"], {
|
||||
**original["components"], "repeater_scope_storage_fallback": scope,
|
||||
"repeater_advert_storage_fallback": advert})
|
||||
self.assertEqual(selected["required_heap_bytes"],
|
||||
original["required_heap_bytes"] + scope + advert)
|
||||
self.assertEqual(selected["mqtt"], original["mqtt"])
|
||||
self.assertEqual(selected["components"]["allocation_and_transient_margin"], 16384)
|
||||
self.assertEqual(selected["components"]["wireless_stacks"], 49152)
|
||||
self.assertEqual(ram.requirements("ESP32_PLATFORM", {
|
||||
**flags, "MESH_REPEATER_PSRAM_TABLES": 1}, target), selected)
|
||||
|
||||
def test_g3_psram_table_defaults_follow_board_psram_platform_and_source_role_gates(self):
|
||||
flags = {"STATION_G3_ESP32": 1, "BOARD_HAS_PSRAM": 1,
|
||||
"MESH_CLIENT_REPEATER_ONLY": 1}
|
||||
cases = [
|
||||
("ESP32_PLATFORM", {k: v for k, v in flags.items() if k != "STATION_G3_ESP32"},
|
||||
"other_s3_repeater"),
|
||||
("ESP32_PLATFORM", {k: v for k, v in flags.items() if k != "BOARD_HAS_PSRAM"},
|
||||
"Station_G3_ESP32_repeater"),
|
||||
("ESP32_PLATFORM", {k: v for k, v in flags.items() if k != "MESH_CLIENT_REPEATER_ONLY"},
|
||||
"Station_G3_ESP32_repeater"),
|
||||
("ESP32_PLATFORM", {**flags, "MESH_CLIENT_REPEATER_ONLY": 0},
|
||||
"Station_G3_ESP32_repeater"),
|
||||
("ESP32_PLATFORM", {**flags, "MESH_CLIENT_REPEATER_ONLY": 0,
|
||||
"COMPANION_RADIO_FULL": 1},
|
||||
"Station_G3_ESP32_companion_radio_full"),
|
||||
("ESP32_PLATFORM", {**flags, "MESH_CLIENT_REPEATER_ONLY": 0},
|
||||
"Station_G3_ESP32_room_server"),
|
||||
]
|
||||
for platform in ("NRF52_PLATFORM", "RP2040_PLATFORM", "STM32_PLATFORM"):
|
||||
cases.extend((platform, {**flags, **extra}, "other_repeater")
|
||||
for extra in ({}, {"MESH_REPEATER_PSRAM_TABLES": 1}))
|
||||
for platform, defines, target in cases:
|
||||
with self.subTest(platform=platform, defines=defines, target=target):
|
||||
policy = ram.requirements(platform, defines, target)
|
||||
self.assertNotIn("repeater_scope_storage_fallback", policy["components"])
|
||||
self.assertNotIn("repeater_advert_storage_fallback", policy["components"])
|
||||
self.assertEqual(policy, ram.requirements(platform, {
|
||||
**defines, "MESH_REPEATER_PSRAM_TABLES": 0}, target))
|
||||
# The source role, rather than a release filename, selects MyMesh.
|
||||
renamed = ram.requirements("ESP32_PLATFORM", flags, "custom_board_forwarder")
|
||||
self.assertIn("repeater_scope_storage_fallback", renamed["components"])
|
||||
self.assertIn("repeater_advert_storage_fallback", renamed["components"])
|
||||
|
||||
def test_explicit_psram_repeater_table_opt_in_and_capacity_overrides_are_fully_budgeted(self):
|
||||
flags = {"BOARD_HAS_PSRAM": 1, "MESH_CLIENT_REPEATER_ONLY": 1,
|
||||
"MESH_REPEATER_PSRAM_TABLES": 1}
|
||||
cases = (
|
||||
({}, 255, 255, 32, 128),
|
||||
({"FLOOD_CHANNEL_SCOPE_SLOTS": 5, "MAX_REGION_ENTRIES": 2}, 5, 5, 2, 128),
|
||||
({"FLOOD_CHANNEL_SCOPE_SLOTS": 7, "FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS": 11,
|
||||
"FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS": 3, "FLOOD_ADVERT_SOURCE_SLOTS": 17},
|
||||
7, 11, 3, 17),
|
||||
({"FLOOD_CHANNEL_SCOPE_SLOTS": 3, "MAX_REGION_ENTRIES": 2,
|
||||
"FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS": 9}, 3, 3, 9, 128),
|
||||
({"MAX_REGION_ENTRIES": 64, "FLOOD_ADVERT_SOURCE_SLOTS": 200},
|
||||
255, 255, 64, 200),
|
||||
)
|
||||
for extra, scopes, required, direct, adverts in cases:
|
||||
with self.subTest(extra=extra):
|
||||
defines = {**flags, **extra}
|
||||
selected = ram.requirements("ESP32_PLATFORM", defines, "other_s3_repeater")
|
||||
original = ram.requirements("ESP32_PLATFORM", {
|
||||
**defines, "MESH_REPEATER_PSRAM_TABLES": 0}, "other_s3_repeater")
|
||||
scope = scopes * 36 + required * 34 + direct * 32 + 2 + 16
|
||||
advert = adverts * 140 + 16
|
||||
self.assertEqual(selected["components"], {
|
||||
**original["components"], "repeater_scope_storage_fallback": scope,
|
||||
"repeater_advert_storage_fallback": advert})
|
||||
self.assertEqual(selected["required_heap_bytes"],
|
||||
original["required_heap_bytes"] + scope + advert)
|
||||
|
||||
def test_psram_advert_budget_excludes_portable_iram_and_rtc_storage_only(self):
|
||||
flags = {"STATION_G3_ESP32": 1, "BOARD_HAS_PSRAM": 1,
|
||||
"MESH_CLIENT_REPEATER_ONLY": 1}
|
||||
target = "Station_G3_ESP32_repeater"
|
||||
original = ram.requirements("ESP32_PLATFORM", flags, target)
|
||||
advert = original["components"]["repeater_advert_storage_fallback"]
|
||||
for extra in ({"PORTABLE_MQTT_OBSERVER": 1}, {"PORTABLE_MQTT_OBSERVER": 0},
|
||||
{"FLOOD_ADVERT_IRAM_WORD_STORAGE": 1},
|
||||
{"FLOOD_ADVERT_RTC_FAST_SLOTS": 16},
|
||||
{"FLOOD_ADVERT_RTC_SLOW_SLOTS": 50},
|
||||
{"FLOOD_ADVERT_RTC_FAST_SLOTS": 50, "FLOOD_ADVERT_RTC_SLOW_SLOTS": 50}):
|
||||
with self.subTest(extra=extra):
|
||||
policy = ram.requirements("ESP32_PLATFORM", {**flags, **extra}, target)
|
||||
self.assertEqual(policy["components"], {
|
||||
k: v for k, v in original["components"].items()
|
||||
if k != "repeater_advert_storage_fallback"})
|
||||
self.assertEqual(policy["required_heap_bytes"],
|
||||
original["required_heap_bytes"] - advert)
|
||||
disabled_backends = {"FLOOD_ADVERT_IRAM_WORD_STORAGE": 0,
|
||||
"FLOOD_ADVERT_RTC_FAST_SLOTS": 0, "FLOOD_ADVERT_RTC_SLOW_SLOTS": 0}
|
||||
self.assertEqual(ram.requirements("ESP32_PLATFORM", {
|
||||
**flags, **disabled_backends}, target), original)
|
||||
|
||||
def test_each_psram_table_fallback_can_set_the_contiguous_allocation_bound(self):
|
||||
flags = {"STATION_G3_ESP32": 1, "BOARD_HAS_PSRAM": 1,
|
||||
"MESH_CLIENT_REPEATER_ONLY": 1, "MAX_CLIENTS": 1,
|
||||
"FLOOD_PACKET_FILTER_SLOTS": 1, "MAX_RECENT_REPEATERS": 0}
|
||||
for extra, name in (({}, "repeater_scope_storage_fallback"),
|
||||
({"FLOOD_ADVERT_SOURCE_SLOTS": 200},
|
||||
"repeater_advert_storage_fallback")):
|
||||
with self.subTest(extra=extra):
|
||||
policy = ram.requirements("ESP32_PLATFORM", {**flags, **extra}, "g3_repeater")
|
||||
self.assertEqual(policy["required_contiguous_bytes"], policy["components"][name])
|
||||
self.assertGreater(policy["required_contiguous_bytes"], 16384)
|
||||
|
||||
def test_g3_psram_fallback_elf_gate_rejects_old_budget_and_enforces_exact_total(self):
|
||||
flags = {"STATION_G3_ESP32": 1, "BOARD_HAS_PSRAM": 1,
|
||||
"MESH_CLIENT_REPEATER_ONLY": 1, "WITH_MQTT_BRIDGE": 1}
|
||||
target = "Station_G3_ESP32_repeater"
|
||||
previous = {**flags, "MESH_REPEATER_PSRAM_TABLES": 0}
|
||||
old = ram.requirements("ESP32_PLATFORM", previous, target)["required_heap_bytes"]
|
||||
required = ram.requirements("ESP32_PLATFORM", flags, target)["required_heap_bytes"]
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
path = Path(temporary) / "firmware.elf"
|
||||
report = Path(temporary) / "firmware.memory.json"
|
||||
for capacity, accepted in ((old, False), (required - 1, False), (required, True)):
|
||||
with self.subTest(capacity=capacity), contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
|
||||
esp_fixture(path, modern=True, internal_length=capacity + 0x4000)
|
||||
self.assertEqual(ram.check_firmware(path, "ESP32_PLATFORM", "esp32s3",
|
||||
previous, target), 0)
|
||||
status = ram.check_firmware(path, "ESP32_PLATFORM", "esp32s3",
|
||||
flags, target, report)
|
||||
result = json.loads(report.read_text())
|
||||
self.assertEqual(status == 0, accepted)
|
||||
self.assertEqual(result["passed"], accepted)
|
||||
self.assertEqual(result["available_internal_bytes"], capacity)
|
||||
self.assertEqual(result["required_heap_bytes"], required)
|
||||
self.assertEqual(result["components"]["repeater_scope_storage_fallback"], 18892)
|
||||
self.assertEqual(result["components"]["repeater_advert_storage_fallback"], 17936)
|
||||
|
||||
def test_browser_terminal_reserves_internal_session_and_psram_aware_scrollback(self):
|
||||
defines = {"ENABLE_USB_INTERFACE": 1, "WIFI_SSID": "", "DISPLAY_CLASS": "SSD1306Display"}
|
||||
base = ram.requirements("ESP32_PLATFORM", {**defines, "WEBCONFIG_DISABLED": 1}, "v4_companion")
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
"""Keep every advert-history decision while moving its complete table to PSRAM."""
|
||||
from pathlib import Path
|
||||
import os
|
||||
import resource
|
||||
import signal
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from test_flood_advert_rtc import HARNESS
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
HEAP = r'''
|
||||
#pragma once
|
||||
#include <cassert>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <map>
|
||||
#define HEAP_CHECK(value) do { if (!(value)) std::abort(); } while (0)
|
||||
enum { MALLOC_CAP_INTERNAL=1, MALLOC_CAP_DMA=2,
|
||||
MALLOC_CAP_SPIRAM=4, MALLOC_CAP_8BIT=8 };
|
||||
namespace heap_test {
|
||||
struct Block { size_t bytes; unsigned caps; };
|
||||
struct Heap {
|
||||
bool psram = true, internal = true;
|
||||
size_t dma_free = 100000, dma_largest = 100000;
|
||||
unsigned external_calls = 0, internal_calls = 0, frees = 0;
|
||||
size_t last_bytes = 0;
|
||||
std::map<void*, Block> blocks;
|
||||
};
|
||||
inline Heap& heap() { static Heap value; return value; }
|
||||
}
|
||||
inline void* heap_caps_malloc(size_t size, unsigned caps) {
|
||||
using namespace heap_test;
|
||||
Heap& state = heap();
|
||||
state.last_bytes = size;
|
||||
const bool external = (caps & MALLOC_CAP_SPIRAM) != 0;
|
||||
if (external) ++state.external_calls; else ++state.internal_calls;
|
||||
if (external ? !state.psram : !state.internal) return nullptr;
|
||||
if (!external) {
|
||||
HEAP_CHECK(caps & MALLOC_CAP_INTERNAL);
|
||||
HEAP_CHECK(size <= state.dma_largest && size <= state.dma_free - 16384);
|
||||
state.dma_free -= size;
|
||||
}
|
||||
void* memory = std::malloc(size);
|
||||
HEAP_CHECK(memory);
|
||||
std::memset(memory, 0xa5, size);
|
||||
state.blocks[memory] = {size, caps};
|
||||
return memory;
|
||||
}
|
||||
inline void* heap_caps_calloc(size_t count, size_t size, unsigned caps) {
|
||||
void* memory = heap_caps_malloc(count * size, caps);
|
||||
if (memory) std::memset(memory, 0, count * size);
|
||||
return memory;
|
||||
}
|
||||
inline void heap_caps_free(void* memory) {
|
||||
if (!memory) return;
|
||||
using namespace heap_test;
|
||||
Heap& state = heap();
|
||||
const auto found = state.blocks.find(memory);
|
||||
HEAP_CHECK(found != state.blocks.end());
|
||||
if (!(found->second.caps & MALLOC_CAP_SPIRAM))
|
||||
state.dma_free += found->second.bytes;
|
||||
state.blocks.erase(found);
|
||||
++state.frees;
|
||||
std::free(memory);
|
||||
}
|
||||
inline size_t heap_caps_get_free_size(unsigned caps) {
|
||||
HEAP_CHECK(caps == (MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA));
|
||||
return heap_test::heap().dma_free;
|
||||
}
|
||||
inline size_t heap_caps_get_largest_free_block(unsigned caps) {
|
||||
HEAP_CHECK(caps == (MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA));
|
||||
return heap_test::heap().dma_largest;
|
||||
}
|
||||
'''
|
||||
|
||||
WRAPPER = r'''
|
||||
int main(int argc, char** argv) {
|
||||
auto& heap = heap_test::heap();
|
||||
const char* mode = argc > 1 ? argv[1] : "trace";
|
||||
const bool fallback = !std::strcmp(mode, "fallback");
|
||||
const bool lifetime = !std::strcmp(mode, "lifetime");
|
||||
if (fallback || !std::strcmp(mode, "no-memory")
|
||||
|| !std::strcmp(mode, "reserve") || !std::strcmp(mode, "fragmented"))
|
||||
heap.psram = false;
|
||||
if (!std::strcmp(mode, "no-memory")) heap.internal = false;
|
||||
if (!std::strcmp(mode, "reserve")) heap.dma_free = 16384 + 17920 - 1;
|
||||
if (!std::strcmp(mode, "fragmented")) heap.dma_largest = 17920 - 1;
|
||||
char trace[] = "trace";
|
||||
char life[] = "lifetime";
|
||||
char* args[] = {argv[0], lifetime ? life : trace, nullptr};
|
||||
const size_t dma_before = heap.dma_free;
|
||||
const int result = traceMain(2, args);
|
||||
CHECK(heap.blocks.empty() && heap.dma_free == dma_before);
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
CHECK(heap.last_bytes == 128 * sizeof(mesh::FloodAdvertLimiter::Entry));
|
||||
CHECK(heap.external_calls == (lifetime ? 3u : 1u));
|
||||
CHECK(heap.internal_calls == (fallback ? 1u : 0u));
|
||||
CHECK(heap.frees == (lifetime ? 3u : 1u));
|
||||
#else
|
||||
CHECK(heap.external_calls == 0 && heap.internal_calls == 0 && heap.frees == 0);
|
||||
#endif
|
||||
return result;
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
class FloodAdvertPsramTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.temp = tempfile.TemporaryDirectory(prefix="advert-psram-")
|
||||
cls.work = Path(cls.temp.name)
|
||||
(cls.work / "esp_heap_caps.h").write_text(HEAP)
|
||||
(cls.work / "sdkconfig.h").write_text(
|
||||
"#define CONFIG_IDF_TARGET_ESP32S3 1\n"
|
||||
"#define CONFIG_SPIRAM_BOOT_INIT 1\n"
|
||||
"#define CONFIG_SPIRAM_USE_MALLOC 1\n")
|
||||
cls.compiler = [os.environ.get("CXX", "g++")]
|
||||
cls.binaries = {}
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.temp.cleanup()
|
||||
|
||||
def build(self, enabled, collisions=False):
|
||||
key = (enabled, collisions)
|
||||
if key in self.binaries:
|
||||
return self.binaries[key]
|
||||
name = "probe-%d-%d" % key
|
||||
source = self.work / (name + ".cpp")
|
||||
binary = self.work / name
|
||||
code = HARNESS.replace("@BANKS@", "").replace("int main(", "int traceMain(")
|
||||
# Release builds of the limiter are exercised while harness assertions
|
||||
# stay active. The trace also checks exact full-key prefix collisions.
|
||||
code = code.rstrip()[:-1] + " return 0;\n}\n"
|
||||
if collisions:
|
||||
code = code.replace("key[0] = uint8_t(i); key[1] = uint8_t(i >> 8);",
|
||||
"key[0] = uint8_t(i / 2); key[1] = uint8_t(i >> 8);")
|
||||
source.write_text("#include <esp_heap_caps.h>\n" + code + WRAPPER)
|
||||
args = ["-std=c++11", "-Os", "-DNDEBUG", "-Wall", "-Wextra", "-Werror",
|
||||
"-fsanitize=address,undefined", "-fno-sanitize-recover=all",
|
||||
"-fno-pie", "-no-pie", "-DESP32_PLATFORM=1", "-DESP_PLATFORM=1",
|
||||
"-DBOARD_HAS_PSRAM=1", "-DSTATION_G3_ESP32=1",
|
||||
"-DMESH_CLIENT_REPEATER_ONLY=1",
|
||||
"-DMESH_REPEATER_PSRAM_TABLES=" + str(enabled),
|
||||
"-I", str(self.work), "-isystem", str(ROOT / "src"),
|
||||
"-isystem", str(ROOT / "test/mocks")]
|
||||
result = subprocess.run(
|
||||
self.compiler + args + [str(source),
|
||||
str(ROOT / "src/helpers/FloodAdvertLimiter.cpp"), "-o", str(binary)],
|
||||
capture_output=True, text=True, timeout=30)
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.binaries[key] = binary
|
||||
return binary
|
||||
|
||||
def run_probe(self, enabled, mode="trace", collisions=False):
|
||||
return subprocess.run([str(self.build(enabled, collisions)), mode],
|
||||
capture_output=True, text=True, timeout=20)
|
||||
|
||||
def test_full_capacity_quota_duplicate_badlist_and_rollover_traces_match(self):
|
||||
for collisions in (False, True):
|
||||
with self.subTest(collisions=collisions):
|
||||
baseline = self.run_probe(0, collisions=collisions)
|
||||
self.assertEqual(baseline.returncode, 0, baseline.stderr)
|
||||
self.assertIn("limited=128", baseline.stdout)
|
||||
for mode in ("trace", "fallback"):
|
||||
result = self.run_probe(1, mode, collisions)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(result.stdout, baseline.stdout)
|
||||
|
||||
def test_owner_recreation_clears_history_and_releases_every_allocation(self):
|
||||
baseline = self.run_probe(0, "lifetime")
|
||||
result = self.run_probe(1, "lifetime")
|
||||
self.assertEqual(baseline.returncode, 0, baseline.stderr)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(result.stdout, baseline.stdout)
|
||||
|
||||
def test_exhausted_or_reserved_internal_memory_stops_startup(self):
|
||||
def no_core():
|
||||
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
|
||||
binary = self.build(1)
|
||||
for mode in ("no-memory", "reserve", "fragmented"):
|
||||
with self.subTest(mode=mode):
|
||||
result = subprocess.run([str(binary), mode], capture_output=True,
|
||||
text=True, preexec_fn=no_core, timeout=20)
|
||||
self.assertEqual(result.returncode, -signal.SIGABRT, result.stderr)
|
||||
self.assertIn("Cannot allocate advert source history (17920 bytes)",
|
||||
result.stderr)
|
||||
self.assertNotIn("runtime error:", result.stderr)
|
||||
self.assertNotIn("AddressSanitizer", result.stderr)
|
||||
|
||||
def test_selected_owner_retains_no_full_internal_fallback(self):
|
||||
source = self.work / "layout.cpp"
|
||||
source.write_text('#include <helpers/FloodAdvertLimiter.h>\n'
|
||||
'static_assert(sizeof(mesh::StaticFloodAdvertLimiter<>) < 256, '
|
||||
'"advert owner must not retain a full inline table");\n')
|
||||
result = subprocess.run(self.compiler + ["-std=c++11", "-fsyntax-only",
|
||||
"-DESP32_PLATFORM=1", "-DESP_PLATFORM=1", "-DBOARD_HAS_PSRAM=1",
|
||||
"-DSTATION_G3_ESP32=1", "-DMESH_CLIENT_REPEATER_ONLY=1",
|
||||
"-DMESH_REPEATER_PSRAM_TABLES=1", "-I", str(self.work),
|
||||
"-isystem", str(ROOT / "src"), "-isystem", str(ROOT / "test/mocks"),
|
||||
str(source)], capture_output=True, text=True, timeout=20)
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,375 @@
|
||||
"""Exercise actual scope table owners, persistence and rollback in both RAM backends."""
|
||||
from pathlib import Path
|
||||
import os
|
||||
import resource
|
||||
import signal
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from cpp_source import body
|
||||
from test_flood_advert_psram import HEAP
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
SDK = ("#define CONFIG_IDF_TARGET_ESP32S3 1\n"
|
||||
"#define CONFIG_SPIRAM_BOOT_INIT 1\n"
|
||||
"#define CONFIG_SPIRAM_USE_MALLOC 1\n")
|
||||
DEFINES = ["ESP32_PLATFORM=1", "BOARD_HAS_PSRAM=1", "STATION_G3_ESP32=1",
|
||||
"MESH_CLIENT_REPEATER_ONLY=1"]
|
||||
|
||||
PRELUDE = r'''
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include <esp_heap_caps.h>
|
||||
#include <helpers/RepeaterPsramStorage.h>
|
||||
#include <helpers/FloodFilterPolicy.h>
|
||||
#define CHECK(value) do { if (!(value)) std::abort(); } while (0)
|
||||
#define FLOOD_CHANNEL_SCOPE_SLOTS 255
|
||||
#define FLOOD_CHANNEL_DIRECT_SCOPE_SLOTS 32
|
||||
#define FLOOD_CHANNEL_SCOPE_REQUIRE_SLOTS 255
|
||||
#define FLOOD_PACKET_FILTER_SCOPE_NAME_LEN 32
|
||||
#define FLOOD_CHANNEL_SCOPE_TXT_ANY 0
|
||||
#define FLOOD_CHANNEL_SCOPE_OTHER_ANY 2
|
||||
#define MESH_ENABLE_FLOOD_RULE_ENGINE 0
|
||||
#define FLOOD_CHANNEL_SCOPE_FILE "/scopes"
|
||||
#define FLOOD_CHANNEL_SCOPE_TEMP_FILE "/scopes.tmp"
|
||||
#define FLOOD_CHANNEL_SCOPE_REQUIRE_FILE "/require"
|
||||
#define FLOOD_CHANNEL_SCOPE_REQUIRE_TEMP_FILE "/require.tmp"
|
||||
struct RegionMap {
|
||||
static bool is_name_char(uint8_t c) { return std::isalnum(c) || c == '_' || c == '-'; }
|
||||
};
|
||||
namespace mesh { struct Utils {
|
||||
static void sha256(void* out, size_t size, const uint8_t*, size_t length) {
|
||||
CHECK(size == 1 && (length == 16 || length == 32));
|
||||
std::memset(out, 0, size);
|
||||
}
|
||||
}; }
|
||||
static unsigned external_io = 0;
|
||||
static void inspectSpan(const void* data, size_t size) {
|
||||
const uintptr_t address = reinterpret_cast<uintptr_t>(data);
|
||||
for (const auto& block : heap_test::heap().blocks) {
|
||||
const uintptr_t start = reinterpret_cast<uintptr_t>(block.first);
|
||||
if (address >= start && address < start + block.second.bytes) {
|
||||
CHECK(size <= start + block.second.bytes - address);
|
||||
if (block.second.caps & MALLOC_CAP_SPIRAM) ++external_io;
|
||||
}
|
||||
}
|
||||
}
|
||||
struct MemoryFS;
|
||||
struct File {
|
||||
MemoryFS* fs = nullptr;
|
||||
std::string path;
|
||||
size_t cursor = 0;
|
||||
bool valid = false;
|
||||
explicit operator bool() const { return valid; }
|
||||
size_t read(uint8_t*, size_t);
|
||||
size_t write(const uint8_t*, size_t);
|
||||
void close() { valid = false; }
|
||||
};
|
||||
struct MemoryFS {
|
||||
std::map<std::string, std::vector<uint8_t>> files;
|
||||
bool fail_rename = false, short_write = false;
|
||||
bool exists(const char* path) { return files.count(path); }
|
||||
bool remove(const char* path) { return files.erase(path); }
|
||||
bool rename(const char* from, const char* to) {
|
||||
if (fail_rename || !files.count(from)) return false;
|
||||
files[to] = files[from]; files.erase(from); return true;
|
||||
}
|
||||
};
|
||||
size_t File::read(uint8_t* out, size_t size) {
|
||||
if (!valid) return 0;
|
||||
const auto& bytes = fs->files[path];
|
||||
size = std::min(size, bytes.size() - cursor);
|
||||
inspectSpan(out, size);
|
||||
if (size) std::memcpy(out, bytes.data() + cursor, size);
|
||||
cursor += size; return size;
|
||||
}
|
||||
size_t File::write(const uint8_t* data, size_t size) {
|
||||
if (!valid) return 0;
|
||||
if (fs->short_write && size) --size;
|
||||
inspectSpan(data, size);
|
||||
auto& bytes = fs->files[path];
|
||||
bytes.insert(bytes.end(), data, data + size); return size;
|
||||
}
|
||||
static File openFloodSettingsRead(MemoryFS* fs, const char* path) {
|
||||
return File{fs, path, 0, fs->exists(path)};
|
||||
}
|
||||
static File openFloodSettingsWrite(MemoryFS* fs, const char* path) {
|
||||
fs->files[path].clear(); return File{fs, path, 0, true};
|
||||
}
|
||||
'''
|
||||
|
||||
SCENARIOS = r'''
|
||||
static std::vector<uint8_t> snapshot(const MyMesh& mesh) {
|
||||
std::vector<uint8_t> bytes;
|
||||
const auto add = [&](const void* data, size_t size) {
|
||||
const auto first = static_cast<const uint8_t*>(data);
|
||||
bytes.insert(bytes.end(), first, first + size);
|
||||
};
|
||||
add(mesh.flood_channel_scopes, sizeof(mesh.flood_channel_scopes));
|
||||
add(mesh.flood_channel_direct_scopes, sizeof(mesh.flood_channel_direct_scopes));
|
||||
add(mesh.flood_channel_scope_requirements, sizeof(mesh.flood_channel_scope_requirements));
|
||||
return bytes;
|
||||
}
|
||||
static void populate(MyMesh& mesh) {
|
||||
for (unsigned i = 0; i < 32; ++i)
|
||||
std::snprintf(mesh.flood_channel_direct_scopes[i], 32, "#Scope%u", i);
|
||||
for (unsigned i = 0; i < 255; ++i) {
|
||||
auto& row = mesh.flood_channel_scopes[i];
|
||||
row.target_id = i % 32 + 1;
|
||||
row.selector = FloodFilterPolicy::encodeChannelScopeTargetSelector(0, true);
|
||||
auto& requirement = mesh.flood_channel_scope_requirements[i];
|
||||
requirement.key_len = 32;
|
||||
std::memset(requirement.secret, uint8_t(i + 1), sizeof(requirement.secret));
|
||||
}
|
||||
}
|
||||
static void exercise(const char* mode) {
|
||||
MemoryFS disk;
|
||||
{
|
||||
MyMesh writer(disk);
|
||||
static_assert(sizeof(writer.flood_channel_scopes) == 255 * 36, "all scope slots");
|
||||
static_assert(sizeof(writer.flood_channel_direct_scopes) == 32 * 32, "all direct names");
|
||||
static_assert(sizeof(writer.flood_channel_scope_requirements) == 255 * 34, "all requirements");
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
static_assert(sizeof(MyMesh) < 256, "no inline fallback arrays");
|
||||
auto& heap = heap_test::heap();
|
||||
CHECK(heap.blocks.size() == 1);
|
||||
CHECK(heap.last_bytes == 18874);
|
||||
const auto caps = heap.blocks.begin()->second.caps;
|
||||
CHECK(caps == (!std::strcmp(mode, "fallback")
|
||||
? (MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT)
|
||||
: (MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT)));
|
||||
const unsigned calls = heap.external_calls + heap.internal_calls;
|
||||
CHECK(writer.flood_channel_storage.begin());
|
||||
CHECK(heap.external_calls + heap.internal_calls == calls);
|
||||
#else
|
||||
CHECK(heap_test::heap().blocks.empty());
|
||||
(void)mode;
|
||||
#endif
|
||||
CHECK(snapshot(writer) == std::vector<uint8_t>(18874, 0));
|
||||
const void* scopes = writer.flood_channel_scopes;
|
||||
const void* direct = writer.flood_channel_direct_scopes;
|
||||
const void* require = writer.flood_channel_scope_requirements;
|
||||
populate(writer);
|
||||
const auto initial = snapshot(writer);
|
||||
CHECK(writer.saveFloodChannelScopes());
|
||||
CHECK(writer.saveFloodChannelScopeRequirements());
|
||||
CHECK(disk.files["/scopes"].size() == 10210);
|
||||
CHECK(disk.files["/require"].size() == 8675);
|
||||
std::vector<uint8_t> expected = {'F','C','S','5',255};
|
||||
const auto* start = reinterpret_cast<const uint8_t*>(writer.flood_channel_scopes);
|
||||
expected.insert(expected.end(), start, start + 9180);
|
||||
expected.push_back(32);
|
||||
start = reinterpret_cast<const uint8_t*>(writer.flood_channel_direct_scopes);
|
||||
expected.insert(expected.end(), start, start + 1024);
|
||||
CHECK(disk.files["/scopes"] == expected);
|
||||
expected = {'F','C','R','1',255};
|
||||
start = reinterpret_cast<const uint8_t*>(writer.flood_channel_scope_requirements);
|
||||
expected.insert(expected.end(), start, start + 8670);
|
||||
CHECK(disk.files["/require"] == expected);
|
||||
const auto saved = disk.files;
|
||||
char reply[160];
|
||||
for (bool short_write : {false, true}) {
|
||||
disk.fail_rename = !short_write; disk.short_write = short_write;
|
||||
writer.deleteFloodChannelScope(".255", reply);
|
||||
CHECK(!std::strcmp(reply, "Err - save failed"));
|
||||
CHECK(snapshot(writer) == initial && disk.files == saved);
|
||||
writer.deleteFloodChannelScopeRequirement(".255", reply);
|
||||
CHECK(!std::strcmp(reply, "Err - save failed"));
|
||||
CHECK(snapshot(writer) == initial && disk.files == saved);
|
||||
writer.deleteFloodChannelScope("all", reply);
|
||||
CHECK(!std::strcmp(reply, "Err - save failed"));
|
||||
CHECK(snapshot(writer) == initial && disk.files == saved);
|
||||
writer.deleteFloodChannelScopeRequirement("all", reply);
|
||||
CHECK(!std::strcmp(reply, "Err - save failed"));
|
||||
CHECK(snapshot(writer) == initial && disk.files == saved);
|
||||
}
|
||||
disk.fail_rename = disk.short_write = false;
|
||||
std::memset(writer.flood_channel_scopes, 0xa5, sizeof(writer.flood_channel_scopes));
|
||||
std::memset(writer.flood_channel_direct_scopes, 0xa5, sizeof(writer.flood_channel_direct_scopes));
|
||||
std::memset(writer.flood_channel_scope_requirements, 0xa5, sizeof(writer.flood_channel_scope_requirements));
|
||||
CHECK(writer.loadFloodChannelScopes());
|
||||
writer.loadFloodChannelScopeRequirements();
|
||||
CHECK(snapshot(writer) == initial);
|
||||
disk.files["/scopes"].pop_back();
|
||||
disk.files["/require"].pop_back();
|
||||
CHECK(!writer.loadFloodChannelScopes());
|
||||
writer.loadFloodChannelScopeRequirements();
|
||||
CHECK(snapshot(writer) == std::vector<uint8_t>(18874, 0));
|
||||
disk.files = saved;
|
||||
CHECK(writer.loadFloodChannelScopes());
|
||||
writer.loadFloodChannelScopeRequirements();
|
||||
CHECK(snapshot(writer) == initial);
|
||||
CHECK(scopes == writer.flood_channel_scopes && direct == writer.flood_channel_direct_scopes
|
||||
&& require == writer.flood_channel_scope_requirements);
|
||||
// Compare the complete production file payloads between inline and PSRAM runs.
|
||||
for (const char* path : {"/scopes", "/require"}) {
|
||||
const auto& bytes = disk.files[path];
|
||||
CHECK(std::fwrite(bytes.data(), 1, bytes.size(), stdout) == bytes.size());
|
||||
}
|
||||
writer.deleteFloodChannelScope(".255", reply);
|
||||
CHECK(!std::strcmp(reply, "OK") && writer.flood_channel_scopes[254].target_id == 0);
|
||||
writer.deleteFloodChannelScopeRequirement(".255", reply);
|
||||
CHECK(!std::strcmp(reply, "OK") && writer.flood_channel_scope_requirements[254].key_len == 0);
|
||||
writer.deleteFloodChannelScope("all", reply);
|
||||
CHECK(!std::strcmp(reply, "OK"));
|
||||
writer.deleteFloodChannelScopeRequirement("all", reply);
|
||||
CHECK(!std::strcmp(reply, "OK"));
|
||||
CHECK(snapshot(writer) == std::vector<uint8_t>(18874, 0));
|
||||
}
|
||||
CHECK(heap_test::heap().blocks.empty());
|
||||
MyMesh recreated(disk);
|
||||
CHECK(snapshot(recreated) == std::vector<uint8_t>(18874, 0));
|
||||
}
|
||||
int main(int argc, char** argv) {
|
||||
const char* mode = argc > 1 ? argv[1] : "trace";
|
||||
auto& heap = heap_test::heap();
|
||||
if (std::strcmp(mode, "trace")) heap.psram = false;
|
||||
if (!std::strcmp(mode, "no-memory")) heap.internal = false;
|
||||
if (!std::strcmp(mode, "reserve")) heap.dma_free = 16384 + 18874 - 1;
|
||||
if (!std::strcmp(mode, "fragmented")) heap.dma_largest = 18874 - 1;
|
||||
if (!std::strcmp(mode, "boundary")) {
|
||||
heap.dma_free = 16384 + 18874;
|
||||
heap.dma_largest = 18874;
|
||||
mode = "fallback";
|
||||
}
|
||||
const size_t free_before = heap.dma_free;
|
||||
exercise(mode);
|
||||
CHECK(heap.blocks.empty() && heap.dma_free == free_before);
|
||||
#if MESH_REPEATER_PSRAM_TABLES
|
||||
CHECK(heap.external_calls == 2 && heap.frees == 2);
|
||||
CHECK(heap.internal_calls == (!std::strcmp(mode, "fallback") ? 2u : 0u));
|
||||
CHECK(external_io > 0 || !std::strcmp(mode, "fallback"));
|
||||
#else
|
||||
CHECK(heap.external_calls == 0 && heap.internal_calls == 0 && heap.frees == 0);
|
||||
#endif
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def production_program():
|
||||
header = (ROOT / "examples/simple_repeater/MyMesh.h").read_text(encoding="ascii")
|
||||
source = (ROOT / "examples/simple_repeater/MyMesh.cpp").read_text(encoding="ascii")
|
||||
location = header.index(" struct FloodChannelStorage {")
|
||||
start = header.rfind("#if MESH_REPEATER_PSRAM_TABLES", 0, location)
|
||||
declarations = header[start:header.index("#if MESH_ENABLE_FLOOD_GROUP_MODERATION", location)]
|
||||
constructor = source.index("MyMesh::MyMesh(")
|
||||
start = source.index("#if MESH_REPEATER_PSRAM_TABLES", constructor)
|
||||
binders = source[start:source.index("#endif", start) + len("#endif")]
|
||||
start = source.index(" memset(flood_channel_scopes, 0, sizeof(flood_channel_scopes));", constructor)
|
||||
resets = source[start:source.index("#if MESH_ENABLE_FLOOD_GROUP_MODERATION", start)]
|
||||
methods = [
|
||||
"bool MyMesh::loadFloodChannelScopes(", "bool MyMesh::saveFloodChannelScopes(",
|
||||
"void MyMesh::loadFloodChannelScopeRequirements(",
|
||||
"bool MyMesh::saveFloodChannelScopeRequirements(",
|
||||
"void MyMesh::deleteFloodChannelScope(",
|
||||
"void MyMesh::deleteFloodChannelScopeRequirement(",
|
||||
]
|
||||
helpers = [
|
||||
"static const char* skipFloodFilterSpaces(", "static bool floodFilterAsciiEqual(",
|
||||
"static bool parseFloodFilterUnsigned(", "static bool parseFloodChannelScopeIndex(",
|
||||
"static bool isValidStoredFloodFilterScopeName(",
|
||||
"static bool isExactFloodChannelScopeSelector(",
|
||||
]
|
||||
owner = "class MyMesh {\npublic:\nMemoryFS* _fs;\n"
|
||||
for name in ("FloodChannelScopeEntry", "FloodChannelScopeRequireEntry"):
|
||||
owner += body(header, "struct " + name + " {") + ";\n"
|
||||
owner += declarations + "\nint end_marker;\n"
|
||||
owner += "explicit MyMesh(MemoryFS& fs) : _fs(&fs),\n" + binders
|
||||
owner += "\nend_marker(0) {\n" + resets + "}\n"
|
||||
owner += ("bool loadFloodChannelScopes();\n"
|
||||
"bool saveFloodChannelScopes(bool empty_table=false);\n"
|
||||
"void loadFloodChannelScopeRequirements();\n"
|
||||
"bool saveFloodChannelScopeRequirements(bool empty_table=false);\n"
|
||||
"void deleteFloodChannelScope(const char*, char*);\n"
|
||||
"void deleteFloodChannelScopeRequirement(const char*, char*);\n};\n")
|
||||
return "\n".join([PRELUDE, owner] + [body(source, item) for item in helpers + methods] + [SCENARIOS])
|
||||
|
||||
|
||||
class RepeaterScopePsramTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.temporary = tempfile.TemporaryDirectory(prefix="repeater-scope-psram-")
|
||||
cls.work = Path(cls.temporary.name)
|
||||
(cls.work / "esp_heap_caps.h").write_text(HEAP, encoding="ascii")
|
||||
(cls.work / "sdkconfig.h").write_text(SDK, encoding="ascii")
|
||||
(cls.work / "scope.cpp").write_text(production_program(), encoding="ascii")
|
||||
cls.compiler = [os.environ.get("CXX", "g++")]
|
||||
cls.binaries = {}
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.temporary.cleanup()
|
||||
|
||||
def build(self, enabled):
|
||||
if enabled not in self.binaries:
|
||||
binary = self.work / ("scope-%d" % enabled)
|
||||
command = self.compiler + ["-std=c++17", "-O1", "-g", "-DNDEBUG",
|
||||
"-Wall", "-Wextra", "-Werror", "-Wno-sign-compare",
|
||||
"-fsanitize=address,undefined", "-fno-sanitize-recover=all",
|
||||
"-fno-pie", "-no-pie", "-I", str(self.work),
|
||||
"-isystem", str(ROOT / "src"), "-isystem", str(ROOT / "test/mocks")]
|
||||
command += ["-D" + define for define in DEFINES]
|
||||
command += ["-DMESH_REPEATER_PSRAM_TABLES=%d" % enabled,
|
||||
str(self.work / "scope.cpp"), "-o", str(binary)]
|
||||
result = subprocess.run(command, capture_output=True, text=True, timeout=60)
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.binaries[enabled] = binary
|
||||
return self.binaries[enabled]
|
||||
|
||||
def test_full_table_file_bytes_loads_resets_lifetime_and_rollback_match(self):
|
||||
baseline = subprocess.run([str(self.build(0))], capture_output=True, timeout=20)
|
||||
self.assertEqual(baseline.returncode, 0, baseline.stderr.decode())
|
||||
self.assertEqual(len(baseline.stdout), 18885)
|
||||
for mode in ("trace", "fallback", "boundary"):
|
||||
with self.subTest(mode=mode):
|
||||
result = subprocess.run([str(self.build(1)), mode], capture_output=True, timeout=20)
|
||||
self.assertEqual(result.returncode, 0, result.stderr.decode())
|
||||
self.assertEqual(result.stdout, baseline.stdout)
|
||||
|
||||
def test_no_capacity_reduction_or_null_references_on_allocation_failure(self):
|
||||
def no_core():
|
||||
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
|
||||
for mode in ("no-memory", "reserve", "fragmented"):
|
||||
with self.subTest(mode=mode):
|
||||
result = subprocess.run([str(self.build(1)), mode], capture_output=True,
|
||||
text=True, timeout=20, preexec_fn=no_core)
|
||||
self.assertEqual(result.returncode, -signal.SIGABRT, result.stderr)
|
||||
self.assertIn("Cannot allocate channel scope tables (18874 bytes)", result.stderr)
|
||||
self.assertNotIn("runtime error:", result.stderr)
|
||||
self.assertNotIn("AddressSanitizer", result.stderr)
|
||||
|
||||
def test_default_selection_and_explicit_sdk_prerequisites(self):
|
||||
probe = self.work / "guard.cpp"
|
||||
probe.write_text('#include <helpers/RepeaterPsramStorage.h>\n'
|
||||
'static_assert(MESH_REPEATER_PSRAM_TABLES == EXPECTED, "selection");\n',
|
||||
encoding="ascii")
|
||||
cases = [([], 0, True, SDK), (["ESP32_PLATFORM=1", "STATION_G3_ESP32=1"], 0, True, SDK),
|
||||
(["ESP32_PLATFORM=1", "BOARD_HAS_PSRAM=1"], 0, True, SDK),
|
||||
(DEFINES[:-1], 0, True, SDK),
|
||||
(DEFINES, 1, True, SDK), (DEFINES + ["MESH_REPEATER_PSRAM_TABLES=0"], 0, True, ""),
|
||||
(["MESH_REPEATER_PSRAM_TABLES=1"], 1, False, SDK),
|
||||
(DEFINES + ["MESH_REPEATER_PSRAM_TABLES=2"], 2, False, SDK),
|
||||
(DEFINES, 1, False, SDK.replace("CONFIG_IDF_TARGET_ESP32S3 1", "CONFIG_IDF_TARGET_ESP32 1")),
|
||||
(DEFINES, 1, False, SDK.replace("CONFIG_SPIRAM_BOOT_INIT 1", "CONFIG_SPIRAM_BOOT_INIT 0")),
|
||||
(DEFINES, 1, False, SDK.replace("CONFIG_SPIRAM_USE_MALLOC 1", "CONFIG_SPIRAM_USE_MALLOC 0"))]
|
||||
for index, (defines, expected, success, config) in enumerate(cases):
|
||||
with self.subTest(index=index, defines=defines):
|
||||
config_dir = self.work / ("guard-%d" % index)
|
||||
config_dir.mkdir()
|
||||
(config_dir / "sdkconfig.h").write_text(config, encoding="ascii")
|
||||
command = self.compiler + ["-std=c++17", "-fsyntax-only", "-I", str(config_dir),
|
||||
"-I", str(self.work), "-I", str(ROOT / "src"), "-DEXPECTED=%d" % expected]
|
||||
command += ["-D" + define for define in defines] + [str(probe)]
|
||||
result = subprocess.run(command, capture_output=True, text=True, timeout=20)
|
||||
self.assertEqual(result.returncode == 0, success, result.stdout + result.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,327 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Exercise the actual WS wrapper with SDK allocation/lifetime stubs."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
HEADERS = {
|
||||
"esp_idf_version.h": "#pragma once\n",
|
||||
"sdkconfig.h": "#pragma once\n#define CONFIG_WS_BUFFER_SIZE 1024\n",
|
||||
"esp_transport.h": """
|
||||
#pragma once
|
||||
#include <stdint.h>
|
||||
struct TestTransport;
|
||||
typedef TestTransport* esp_transport_handle_t;
|
||||
extern "C" void* esp_transport_get_context_data(esp_transport_handle_t);
|
||||
extern "C" int esp_transport_destroy(esp_transport_handle_t);
|
||||
""",
|
||||
"esp_transport_ws.h": '#pragma once\n#include "esp_transport.h"\n',
|
||||
"esp_heap_caps.h": """
|
||||
#pragma once
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
#define MALLOC_CAP_SPIRAM 1U
|
||||
#define MALLOC_CAP_8BIT 2U
|
||||
#define MALLOC_CAP_INTERNAL 4U
|
||||
#define MALLOC_CAP_DMA 8U
|
||||
void* heap_caps_malloc(size_t, uint32_t);
|
||||
size_t heap_caps_get_free_size(uint32_t);
|
||||
size_t heap_caps_get_largest_free_block(uint32_t);
|
||||
""",
|
||||
}
|
||||
|
||||
HARNESS = r"""
|
||||
#include <cassert>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <string>
|
||||
#include <stdlib.h>
|
||||
#include "esp_heap_caps.h"
|
||||
|
||||
static void* padded_malloc(size_t);
|
||||
static void sdk_free(void*);
|
||||
#define malloc padded_malloc
|
||||
#define free sdk_free
|
||||
#include "src/helpers/ESP32WsTransportFix.cpp"
|
||||
#undef free
|
||||
#undef malloc
|
||||
|
||||
struct TestTransport {
|
||||
void* context;
|
||||
};
|
||||
|
||||
#if ESP_IDF_VERSION_MAJOR == 4
|
||||
using TestWs = transport_ws_t;
|
||||
#else
|
||||
struct TestWs {
|
||||
char* path;
|
||||
char* buffer;
|
||||
char* sub_protocol;
|
||||
char* user_agent;
|
||||
char* headers;
|
||||
esp_transport_handle_t parent;
|
||||
};
|
||||
#endif
|
||||
|
||||
enum Kind { SDK, GENERIC, EXTERNAL, INTERNAL };
|
||||
struct Allocation {
|
||||
void* ptr;
|
||||
size_t size;
|
||||
Kind kind;
|
||||
};
|
||||
static Allocation allocations[32];
|
||||
static size_t allocation_count;
|
||||
static unsigned destroy_calls;
|
||||
static unsigned original_frees;
|
||||
static unsigned generic_calls;
|
||||
static unsigned external_calls;
|
||||
static unsigned internal_calls;
|
||||
static unsigned free_queries;
|
||||
static unsigned largest_queries;
|
||||
static bool fail_sdk;
|
||||
static bool fail_external;
|
||||
static bool fail_internal;
|
||||
static bool fail_generic;
|
||||
static size_t dma_free = 65536;
|
||||
static size_t dma_largest = 8192;
|
||||
static char* original;
|
||||
static TestTransport parent;
|
||||
|
||||
static void* allocate(size_t bytes, Kind kind) {
|
||||
assert(allocation_count < 32);
|
||||
void* p = std::calloc(1, bytes);
|
||||
assert(p);
|
||||
allocations[allocation_count++] = {p, bytes, kind};
|
||||
return p;
|
||||
}
|
||||
|
||||
static Allocation lookup(void* ptr) {
|
||||
for (size_t i = 0; i < allocation_count; ++i) {
|
||||
if (allocations[i].ptr == ptr) return allocations[i];
|
||||
}
|
||||
std::abort();
|
||||
}
|
||||
|
||||
static void sdk_free(void* ptr) {
|
||||
if (!ptr) return;
|
||||
for (size_t i = 0; i < allocation_count; ++i) {
|
||||
if (allocations[i].ptr != ptr) continue;
|
||||
if (ptr == original) ++original_frees;
|
||||
allocations[i] = allocations[--allocation_count];
|
||||
std::free(ptr);
|
||||
return;
|
||||
}
|
||||
std::abort(); // Double free, wrong allocator, or parent destruction.
|
||||
}
|
||||
|
||||
static void* padded_malloc(size_t bytes) {
|
||||
++generic_calls;
|
||||
assert(bytes == 1025);
|
||||
return fail_generic ? nullptr : allocate(bytes, GENERIC);
|
||||
}
|
||||
|
||||
void* heap_caps_malloc(size_t bytes, uint32_t caps) {
|
||||
assert(bytes == 1025);
|
||||
if (caps == (MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT)) {
|
||||
++external_calls;
|
||||
return fail_external ? nullptr : allocate(bytes, EXTERNAL);
|
||||
}
|
||||
assert(caps == (MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA | MALLOC_CAP_8BIT));
|
||||
++internal_calls;
|
||||
assert(dma_free >= 16384 + bytes && dma_largest >= bytes);
|
||||
return fail_internal ? nullptr : allocate(bytes, INTERNAL);
|
||||
}
|
||||
|
||||
size_t heap_caps_get_free_size(uint32_t caps) {
|
||||
assert(caps == (MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA));
|
||||
++free_queries;
|
||||
return dma_free;
|
||||
}
|
||||
|
||||
size_t heap_caps_get_largest_free_block(uint32_t caps) {
|
||||
assert(caps == (MALLOC_CAP_INTERNAL | MALLOC_CAP_DMA));
|
||||
++largest_queries;
|
||||
return dma_largest;
|
||||
}
|
||||
|
||||
extern "C" void* esp_transport_get_context_data(esp_transport_handle_t t) {
|
||||
assert(t && t != &parent);
|
||||
return t->context;
|
||||
}
|
||||
|
||||
extern "C" int esp_transport_destroy(esp_transport_handle_t t) {
|
||||
assert(t && t != &parent);
|
||||
++destroy_calls;
|
||||
TestWs* ws = static_cast<TestWs*>(t->context);
|
||||
assert(ws->parent == &parent);
|
||||
sdk_free(ws->buffer);
|
||||
sdk_free(ws->path);
|
||||
sdk_free(ws->sub_protocol);
|
||||
sdk_free(ws->user_agent);
|
||||
sdk_free(ws->headers);
|
||||
sdk_free(ws);
|
||||
sdk_free(t);
|
||||
return 0;
|
||||
}
|
||||
|
||||
extern "C" esp_transport_handle_t __real_esp_transport_ws_init(esp_transport_handle_t p) {
|
||||
assert(p == &parent);
|
||||
if (fail_sdk) return nullptr;
|
||||
auto* t = static_cast<TestTransport*>(allocate(sizeof(TestTransport), SDK));
|
||||
auto* ws = static_cast<TestWs*>(allocate(sizeof(TestWs), SDK));
|
||||
t->context = ws;
|
||||
ws->parent = p;
|
||||
ws->path = static_cast<char*>(allocate(2, SDK));
|
||||
ws->sub_protocol = static_cast<char*>(allocate(5, SDK));
|
||||
ws->user_agent = static_cast<char*>(allocate(8, SDK));
|
||||
ws->headers = static_cast<char*>(allocate(8, SDK));
|
||||
ws->buffer = static_cast<char*>(allocate(1024, SDK));
|
||||
original = ws->buffer;
|
||||
return t;
|
||||
}
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
assert(argc == 2);
|
||||
const std::string scenario(argv[1]);
|
||||
bool expect_failure = false;
|
||||
Kind expected = GENERIC;
|
||||
if (scenario == "sdk_failure") {
|
||||
fail_sdk = true;
|
||||
expect_failure = true;
|
||||
} else if (scenario == "external_success") {
|
||||
// External success must not inspect or consume scarce internal memory.
|
||||
dma_free = 0;
|
||||
dma_largest = 0;
|
||||
expected = EXTERNAL;
|
||||
} else if (scenario == "internal_boundary") {
|
||||
fail_external = true;
|
||||
dma_free = 16384 + 1025;
|
||||
dma_largest = 1025;
|
||||
expected = INTERNAL;
|
||||
} else if (scenario == "reserve_denied") {
|
||||
fail_external = true;
|
||||
dma_free = 16384 + 1024;
|
||||
expect_failure = true;
|
||||
} else if (scenario == "largest_denied") {
|
||||
fail_external = true;
|
||||
dma_largest = 1024;
|
||||
expect_failure = true;
|
||||
} else if (scenario == "both_fail") {
|
||||
fail_external = true;
|
||||
fail_internal = true;
|
||||
expect_failure = true;
|
||||
} else if (scenario == "generic_failure") {
|
||||
fail_generic = true;
|
||||
expect_failure = true;
|
||||
} else {
|
||||
assert(scenario == "generic_success" || scenario == "idf5_passthrough");
|
||||
}
|
||||
|
||||
esp_transport_handle_t t = __wrap_esp_transport_ws_init(&parent);
|
||||
if (expect_failure) {
|
||||
assert(t == nullptr);
|
||||
assert(allocation_count == 0);
|
||||
assert(destroy_calls == (fail_sdk ? 0U : 1U));
|
||||
assert(original_frees == (fail_sdk ? 0U : 1U));
|
||||
} else {
|
||||
assert(t && destroy_calls == 0);
|
||||
auto* ws = static_cast<TestWs*>(t->context);
|
||||
Allocation a = lookup(ws->buffer);
|
||||
if (scenario == "idf5_passthrough") {
|
||||
assert(ws->buffer == original && a.size == 1024 && a.kind == SDK);
|
||||
assert(original_frees == 0);
|
||||
} else {
|
||||
assert(ws->buffer != original && a.size == 1025 && a.kind == expected);
|
||||
assert(original_frees == 1);
|
||||
// Exercise the exact off-by-one index the workaround must retain.
|
||||
ws->buffer[1024] = '\0';
|
||||
}
|
||||
esp_transport_destroy(t);
|
||||
assert(allocation_count == 0 && destroy_calls == 1 && original_frees == 1);
|
||||
}
|
||||
if (scenario == "external_success") {
|
||||
assert(external_calls == 1 && internal_calls == 0 && generic_calls == 0);
|
||||
assert(free_queries == 0 && largest_queries == 0);
|
||||
} else if (scenario == "reserve_denied" || scenario == "largest_denied") {
|
||||
assert(external_calls == 1 && internal_calls == 0 && generic_calls == 0);
|
||||
assert(free_queries == 1 && largest_queries == 1);
|
||||
} else if (scenario == "internal_boundary" || scenario == "both_fail") {
|
||||
assert(external_calls == 1 && internal_calls == 1 && generic_calls == 0);
|
||||
assert(free_queries == 1 && largest_queries == 1);
|
||||
} else if (scenario == "generic_success" || scenario == "generic_failure") {
|
||||
assert(external_calls == 0 && internal_calls == 0 && generic_calls == 1);
|
||||
} else {
|
||||
assert(external_calls == 0 && internal_calls == 0 && generic_calls == 0);
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
class WsTransportPsramTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
compiler = os.environ.get("CXX") or shutil.which("g++") or shutil.which("clang++")
|
||||
if not compiler:
|
||||
raise unittest.SkipTest("C++ compiler unavailable")
|
||||
cls.temp = tempfile.TemporaryDirectory(prefix="ws-transport-psram-")
|
||||
cls.addClassCleanup(cls.temp.cleanup)
|
||||
work = Path(cls.temp.name)
|
||||
for name, contents in HEADERS.items():
|
||||
(work / name).write_text(contents, encoding="ascii")
|
||||
source = work / "harness.cpp"
|
||||
source.write_text(HARNESS, encoding="ascii")
|
||||
cls.binaries = {}
|
||||
variants = {
|
||||
"g3_psram": (4, ["STATION_G3_ESP32", "BOARD_HAS_PSRAM"]),
|
||||
"other_psram": (4, ["BOARD_HAS_PSRAM"]),
|
||||
"g3_internal": (4, ["STATION_G3_ESP32"]),
|
||||
"other_internal": (4, []),
|
||||
"idf5": (5, ["STATION_G3_ESP32", "BOARD_HAS_PSRAM"]),
|
||||
}
|
||||
for name, (idf, macros) in variants.items():
|
||||
binary = work / name
|
||||
command = [
|
||||
compiler, "-std=c++17", "-O1", "-g", "-Wall", "-Wextra", "-Werror",
|
||||
"-Wno-unused-function", "-DESP_PLATFORM", f"-DESP_IDF_VERSION_MAJOR={idf}",
|
||||
*[f"-D{macro}" for macro in macros], "-I", str(work), "-I", str(ROOT),
|
||||
]
|
||||
if os.name != "nt":
|
||||
command += ["-fsanitize=address,undefined", "-fno-sanitize-recover=all",
|
||||
"-fno-pie", "-no-pie"]
|
||||
built = subprocess.run([*command, str(source), "-o", str(binary)],
|
||||
capture_output=True, text=True)
|
||||
if built.returncode:
|
||||
raise RuntimeError(built.stdout + built.stderr)
|
||||
cls.binaries[name] = binary
|
||||
|
||||
def run_scenario(self, variant, scenario):
|
||||
result = subprocess.run([str(self.binaries[variant]), scenario],
|
||||
capture_output=True, text=True)
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
|
||||
def test_g3_psram_and_guarded_failure_lifecycles(self):
|
||||
for scenario in ["external_success", "internal_boundary", "reserve_denied",
|
||||
"largest_denied", "both_fail", "sdk_failure"]:
|
||||
with self.subTest(scenario=scenario):
|
||||
self.run_scenario("g3_psram", scenario)
|
||||
|
||||
def test_other_target_allocation_defaults_and_failure_cleanup(self):
|
||||
for variant in ["other_psram", "g3_internal", "other_internal"]:
|
||||
for scenario in ["generic_success", "generic_failure", "sdk_failure"]:
|
||||
with self.subTest(variant=variant, scenario=scenario):
|
||||
self.run_scenario(variant, scenario)
|
||||
|
||||
def test_idf5_is_unchanged_passthrough(self):
|
||||
self.run_scenario("idf5", "idf5_passthrough")
|
||||
self.run_scenario("idf5", "sdk_failure")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user