velxio/backend/app/services
David Montero Crespo a3f21a218e fix(esp32): trim flash image before serializing, pad on QEMU attach
Issue #101 reproducer: an ESP32 sketch that pulls in Adafruit_SSD1306 +
Adafruit_GFX produced a "No response from server. Is the backend
running on port 8001?" error in the browser. The compile actually
succeeded backend-side, but the JSON response carrying the firmware
was ~5.5 MB of base64 — the ESP-IDF compiler builds a full 4 MB merged
flash image (mostly 0xFF padding), encodes it whole, and ships it. In
prod that response goes through nginx + Cloudflare, which buffer-fail
or RST the connection on payloads that big — axios then lands in the
"no response" branch with no HTTP status to surface.

Fix: trim the trailing 0xFF padding before serializing, re-pad to a
valid QEMU flash size (2/4/8/16 MB) just before mtd attach. Lossless:
bytes after `last_used` in the merge are 0xFF by construction, so
trim → pad reproduces the original image byte-for-byte.

Numbers from the reproducer (Adafruit_SSD1306 + Adafruit_GFX,
esp32:esp32:esp32 board):
  before: ~5.5 MB JSON response
  after:  539 KB JSON response (10× smaller)

backend/app/services/espidf_compiler.py
  _merge_flash_image now tracks `last_used` across the three placed
  sections (bootloader / partitions / app) and writes only
  flash[:last_used] to merged_flash.bin.

backend/app/services/esp32_flash_image.py (new)
  Shared `pad_to_flash_size(bytes) -> bytes` helper. Rounds up to the
  next valid QEMU flash size with a 4 MB minimum, matches the
  frontend's existing padToFlashSize logic in Esp32MicroPythonLoader.
  Raises ValueError on >16 MB inputs (would indicate a broken upstream
  merge, not anything user-recoverable).

backend/app/services/esp32_lib_bridge.py
backend/app/services/esp32_worker.py
  Both QEMU consumer paths (in-process and subprocess) call
  pad_to_flash_size right after base64.b64decode, before writing the
  tmp .bin that QEMU attaches with `-drive if=mtd,format=raw`.

Verified: smoke test confirms trim → pad → original is byte-exact.
Edge cases covered: small payloads pad up to the 4 MB minimum;
firmwares >16 MB are rejected loudly.

Closes #101

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 14:36:34 -03:00
..
esp-idf-template feat: ESP32-CAM emulation with webcam frame bridge 2026-05-02 19:29:01 -03:00
picow_net Implement TCP and UDP NAT services for chip-initiated connections 2026-04-29 08:43:41 -03:00
__init__.py feat: Implement Arduino Simulator with component management and simulation features 2026-03-03 00:20:49 -03:00
arduino_cli.py feat: Add library version management and uninstall functionality 2026-05-02 12:54:09 +05:30
chip_compile.py Add ESP32 chip demos and comprehensive tests for I2C, SPI, and UART interactions 2026-04-28 19:24:39 -03:00
esp32_flash_image.py fix(esp32): trim flash image before serializing, pad on QEMU attach 2026-05-08 14:36:34 -03:00
esp32_i2c_slaves.py feat: add BMP280 sensor component and circuit preview 2026-04-14 17:27:30 -03:00
esp32_lib_bridge.py fix(esp32): trim flash image before serializing, pad on QEMU attach 2026-05-08 14:36:34 -03:00
esp32_lib_manager.py feat: ESP32-CAM emulation with webcam frame bridge 2026-05-02 19:29:01 -03:00
esp32_spi_slaves.py feat: add support for UC8159c (ACeP 7-colour) display 2026-04-30 00:27:40 -03:00
esp32_worker.py fix(esp32): trim flash image before serializing, pad on QEMU attach 2026-05-08 14:36:34 -03:00
esp_qemu_manager.py feat: implement ESP32 QEMU backend manager and frontend simulation interface 2026-04-01 22:41:52 -03:00
espidf_compiler.py fix(esp32): trim flash image before serializing, pad on QEMU attach 2026-05-08 14:36:34 -03:00
gpio_shim.py feat: add board types and mappings for Raspberry Pi 3B 2026-03-12 23:39:04 -03:00
metrics.py feat(metrics): add usage analytics dashboard with country tracking 2026-04-25 19:47:40 -03:00
picow_net_bridge.py Implement TCP and UDP NAT services for chip-initiated connections 2026-04-29 08:43:41 -03:00
project_files.py feat: persist multi-board projects + add auto-save 2026-05-01 13:43:33 -03:00
qemu_manager.py feat: Implement Virtual File System for Raspberry Pi 3 with context menu and upload functionality 2026-03-15 14:04:01 -03:00
wasm_chip_runtime.py Add ESP32 chip demos and comprehensive tests for I2C, SPI, and UART interactions 2026-04-28 19:24:39 -03:00
wasm_chip_slave.py Add ESP32 chip demos and comprehensive tests for I2C, SPI, and UART interactions 2026-04-28 19:24:39 -03:00
wifi_status_parser.py feat: implement ESP32 QEMU backend manager and frontend simulation interface 2026-04-01 22:41:52 -03:00