_idf_target mapped only c3/c6/s3 and defaulted everything else to
'esp32', so a P4 fqbn compiled with the XTENSA toolchain and
-DARDUINO_ESP32P4_DEV. Adds p4/c5 predicates, IDF5-target membership,
0x2000 bootloader offsets, the riscv toolchain requirement, PSRAM held
off until the engines validate it, P4 CPU-freq drops (its kconfig
choices do not overlap the classic set), a rev0 sdkconfig pin for the P4
(the in-browser engine runs the only published mask ROM and v3 firmware
reads a flash descriptor rev0 never wrote), and an honest refusal for C5
arduino builds (no core exists). Validated on staging: an esp32p4
arduino sketch now produces a merged image with the riscv32 toolchain
that the esp32p4js engine accepts.
arduino-cli reports errors as plain text or, with --format json, as a JSON
object on stderr. We forwarded that blob verbatim, so on a fresh install
with no library index yet (and no way through the firewall/proxy to
downloads.arduino.cc) the in-app agent told the user "the library server
returns a JSON error" ten times in a row.
humanize_cli_error() maps the known shapes — index download failed,
library zip download failed, library not found — to a single sentence
that says what to check, and keeps the raw text under raw_error. Applied
to search_libraries (rc != 0 and non-JSON stdout) and install_library.
Tests use the real arduino-cli 1.0.4 output captured offline.
ESP-IDF compiles with -ggdb, and for debug builds ccache folds the cwd into
every cache key by default — CCACHE_BASEDIR does not exempt it (verified with
CCACHE_DEBUG on 4.11: 'Hashing CWD /var/lib/velxio-build/esp32s3/v_<hash>/
project/build'). Each build variant lives in its own dir, so every variant had
a private key space: eight esp32 variants of the SAME board each recompiled
all ~2100 IDF objects from scratch, the cache filled with dead per-directory
entries, and a cold Cardputer/S3 build (2263 steps) overran the 600s ninja
timeout on a busy box. CCACHE_NOHASHDIR=1 makes variants of the same
board/sdkconfig share objects; a cold variant becomes mostly cache hits. Only
DW_AT_comp_dir in .debug_info of a reused object can differ, which never
reaches the flashed .bin.
The timeout paths were silent server-side; they now log the build dir and how
many ninja steps completed, and the user-facing message says the partial
build is kept and a retry resumes.
Measured against the user corpus, the by-resolved-type long tail was
losing what the old catch-all happened to include: for `Adafruit_SH1106G
display(...); display.setTextSize(` the class itself declares one method
(begin) and everything people call lives four levels up (Adafruit_SH110X ->
Adafruit_GrayOLED -> Adafruit_GFX -> Print). Same story for MCUFRIEND_kbv,
Adafruit_NeoMatrix and every GFX-derived display driver.
Three changes:
- The scanner records `bases` on class symbols (access/virtual keywords
dropped, `ns::Base` -> Base, template args cut). 2,529 classes across the
production cache now carry their bases.
- /type/{name} walks the chain breadth-first through the type index (bases
routinely live in a different library), re-owns inherited members to the
requested type so the editor sees one flat class, tags them with
`inheritedFrom`, and lists the chain in `bases` -- including bases NOT in
the cache (Print, Stream, or an embedded one like Adafruit_GFX) so the
host can supply those itself. Depth capped at 8. Adafruit_SH1106G goes
from 1 member to 64.
- `void a(int), b(), c(char);` declares three methods; the extractor took
only the first. TFT_eSPI declares setTextColor/setTextSize and
setCursor/getCursorX/... this way. Split on top-level commas after a
complete `name(args)` group. TFT_eSPI 118 -> 173 members.
Rescan of the 1,125-entry production cache: +274 member symbols, 15
libraries gain, none lose.
_scan_text applied `public:` / `private:` / `protected:` only inside
handle_statement, which the `{` branch skipped whenever the statement opened
a nested enum, struct or class. So `public:\n typedef enum { ... }` -- the
first thing in AccelStepper's class body, and a common idiom -- took the enum
branch, the label was never seen, the class stayed in its default private
access, and every method after the enum was dropped. AccelStepper shipped 4
of its public methods; DallasTemperature, SoftWire, MFRC522_I2C, RTCZero
shipped none.
Labels are now consumed before the branch. Rescanned all 1,125 entries of
the production library cache: 82 libraries gain members (+1,532 total,
DallasTemperature 0 -> 57), and the 20 that "lose" one or two were leaking
private/protected members past the old check (PMS::loop is under private:,
hd44780::waitReady under protected:) -- precision, not regression.
The custom sketch.ino.cpp path never ran arduino-cli's ctags prototype
pass, so the universal Arduino idiom (helpers defined after loop()) died
with "'foo' was not declared in this scope" on every ESP32 build. A
conservative generator (skips templates, default args, methods,
function-pointer params) injects prototypes before the first definition
with a #line directive so error line numbers stay put. 8 unit tests.
IDF 5.5 renamed ESP_TASK_WDT to the EN/INIT pair; the v4.4 template's =n
became EN=n, compiling the whole implementation out — every sketch using
esp_task_wdt_add/reset (Firebase's client does internally) failed at
link. The v5 fixup now emits EN=y INIT=n: the API links, the watchdog
never auto-starts under emulation. v4.4 semantics untouched.
GET /api/intellisense/symbols/{Name@Version} scans the content-addressed
library cache entry's headers (dependency-free C++ scanner) into an editor
symbol table, disk-cached per entry and served with long cache headers.
Plain OSS hosts without a lib cache 404 and the editor degrades gracefully.
C3/S3/C6 default to CONFIG_BT_BLE_50_FEATURES_SUPPORTED with 4.2 OFF, so
arduino-esp32's Bluedroid BLE library (esp_ble_gap_config_adv_data_raw,
esp_ble_gap_start_advertising) died at link with undefined references —
esp32c3-ble-advertise never compiled. Enabling 42+50 together matches
arduino-esp32's own per-target sdkconfigs; on classic ESP32 these flags
are already the Kconfig defaults. Found by the nightly gallery smoke.
The dead-branch pruner marked any assumed-live branch as 'taken', so the
#else of an UNKNOWN condition was skipped. GxEPD2 guards its GFX base
class with '#if ENABLE_GxEPD2_GFX (default 0) ... #else #include
<Adafruit_GFX.h>' — the pruned #else silently dropped the Adafruit_GFX
dependency and every GxEPD2 e-paper example failed with 'fatal error:
Adafruit_GFX.h: No such file'. Found by the nightly gallery compile smoke.
'taken' is now set only by PROVABLY-true conditions (all identifiers in
the known-true set), keeping the 2026-08-03 ESPAsyncWebServer behaviour:
provably-dead branches still prune, provably-true branches still kill
their #else.
The main component had them; the merged user-libraries component did not, so
a warning INSIDE a vendor library was still fatal. M5Stack's MahonyAHRS.cpp
(fast inverse square root, a float read through a long*) fails under
-Werror=uninitialized, which made installing the vendor's own library kill
every build using it. Library code the user cannot edit must not stop their
build; the diagnostics still print as warnings.
The library's fast inverse square root reads a float through a long*, GCC
flags y as used uninitialized, and -Werror turns a vendor library the user
cannot edit into a failed build. Demoted to a warning like its siblings.
Two of M5Stack's own published Cardputer examples fail to build here and
compile fine in the Arduino IDE: SSHClient passes an int to a %lu, and
mic_wav_record has a local GCC thinks may be used uninitialized. ESP-IDF
promotes both categories to errors, which is right for IDF itself and wrong
for a sketch a user pasted from the vendor's repository — the emulator looks
broken for code that is not.
format and maybe-uninitialized join the existing relaxation list, which
already carries comment, parentheses, sign-compare and the rest for exactly
this reason. They stay warnings, so the diagnostic is still printed; -Wall is
untouched.
Declared inside the has_arduino branch, so every successful pure
ESP-IDF (language=espidf) build crashed with UnboundLocalError when the
success return read it. Regression from today's manifest auto-migration
change; the Arduino paths were live-verified but the espidf mode was
not.
File names may now carry '/' paths; the explorer renders each board group
as a collapsible folder tree (folders first, files in group order, CSS-var
indentation because .fe-file-item pins padding with !important). New-folder
buttons on the board header and per-folder hover actions (new file here,
new subfolder, delete with confirm when files are inside; a folder holding
every file of the group cannot be deleted - groups keep >= 1 file).
Model: names are the source of truth; only EMPTY folders are tracked
(folderGroups per group) so they survive until a file lands in them, then
materialise into the name prefix. Round-trips through .vlx (optional
folderGroups field, laxly validated), the auth stash draft, and
loadProjectState. Examples gain boards[].files so a single board can ship
a full multi-file tree (first entry = main sketch).
Backend: the arduino-cli sketch writer now creates subdirectories for
'/'-pathed names and REJECTS names that resolve outside the sketch dir
('..' traversal) - previously it crashed on the former and wrote outside
the tmp dir on the latter.
Three-part fix for the library-contamination class of ESP32 compile
failures (a user's project kept pulling unrelated libraries because 79%
of projects compile with an empty manifest, forcing scan-all resolution
over ~1000 shared libraries):
1. Name-affinity ranking in _find_library_for_header: the provider whose
name matches the header stem wins (exact normalized match > esp32
arch declaration > contains-stem, alphabetical only as tie-break).
Previously first-alphabetical won, so any stray library shipping a
generic header (Servo.h has 6 providers) could hijack the build.
2. Manifest auto-migration: scan-all compiles now report the libraries
the build really used (manifest_incomplete +
manifest_suggested_libraries on the compile result); the editor folds
single-candidate suggestions into the board's declared manifest after
a green build, so the NEXT compile runs scoped. One green build = one
migrated project; empty-manifest scan-all behaviour is untouched.
3. Install-declares: the Wokwi import library modal records successful
installs in the active board's manifest, and .vlx export/import now
round-trips board.libraries.
Tests: test_espidf_library_ranking.py (9) + library-manifest.test.ts (19).
Reported 2026-08-03: every ESP32 build in a user's project failed with
errors from libraries he never installed, emitted from user_libs_all --
a folder his file list does not even show, in projects he had recreated
from scratch.
Root cause: _detect_external_includes was a bare regex with no
preprocessor state. ESPAsyncWebServer guards <Hash.h> behind
'#if defined(ESP8266) || defined(TARGET_RP2040)', so the compiler never
sees it on ESP32 -- but the scanner reported it anyway. The resolver
then searched the ~1000 shared-cache libraries, matched stemi-hexapod's
src/Hash.h (a hexapod-robot library), merged it WHOLE, and compiled all
of its sources; each failed on its own unrelated includes.
The scanner now tracks #if/#ifdef/#ifndef/#elif/#else/#endif nesting and
strips comments. A branch is skipped ONLY when every identifier in its
condition is a platform macro known to be undefined on ESP32; unknown
macros keep the branch LIVE, because hiding a genuinely needed header
would be a much worse failure than the old over-reporting. Verified
against the real ESPAsyncWebServer shape plus nesting/elif/else/comment
cases (9 new tests, 15 green in the espidf suites).
Complements the platform-header guard from the previous commit, which
stays as defence in depth.
Reported by a user whose every ESP32 build failed with errors from
libraries he never installed (SoftwareSerial.h, ArduinoBearSSL.h,
Adafruit_GFX.h, AsyncElegantOTA.h) coming out of user_libs_all, in a
project whose file list doesn't even show that folder.
Traced end to end: ESPAsyncWebServer includes <Hash.h> (an ESP8266-core
header, guarded for that platform, so arduino-esp32's tree has no such
file). The global scan then resolved it to stemi-hexapod's src/Hash.h --
a hexapod-robot library -- merged the WHOLE library into user_libs_all,
and its Serial.cpp / Hash.cpp / Server.cpp / BluetoothLowEnergy.cpp /
ExpansionDriver.cpp were compiled, each failing on its own unrelated
includes. Log line: 'Merging "stemihexapod" into user_libs_all for
<Hash.h>'.
Hash.h and friends now join the never-resolve-against-user-libs set.
They cannot be discovered by the core-tree scan (the core genuinely
lacks them) yet their names are generic enough that some installed
library owns a file by that name.
Installing ArduinoJson made the project stop compiling, with an error that
never mentioned ArduinoJson:
CMake Error at user_libs/user_libs_all/CMakeLists.txt:15
(target_compile_definitions):
target_compile_definitions may only set INTERFACE properties on
INTERFACE targets
-- Configuring incomplete, errors occurred!
ArduinoJson ships headers only. With no sources, idf_component_register()
builds the merged user-libs component as an INTERFACE library, and PRIVATE
scope on an INTERFACE target is a hard CMake error - so the configure step
died before a single file was compiled. The user saw only a failing build
and reported it as "ArduinoJson can't be found even after it has been
installed" (#124).
Those definitions map alloca/memcpy_P/memcmp_P for library C sources that
use them undeclared; with no sources there is nothing to define, so they are
emitted only when the component actually compiles something. A library that
carries its own .cpp is unchanged - which is why LiquidCrystal_I2C (#257)
worked and hid this.
Two-tier rule replacing the blanket skip:
- 'all' now counts as wildcard ('*'): LiquidCrystal_I2C 2.0.0 declares
architectures=all and was skipped on every ESP32 build, breaking the
most popular I2C LCD library with a bare 'No such file or directory'.
- Headers the SKETCH includes directly merge even when the library
declares a foreign-only architecture (many avr-declared libs are pure
Wire/SPI code that compiles fine; a truthful compile error beats a
missing-header one). Transitive pulls keep the HARD guard, which is
the path that dragged SAMD-only Adafruit_ZeroDMA into ESP32 builds.
Cada guest es un proceso QEMU con su propia RAM (1-2 GB segun placa) y
sus hilos de vCPU, asi que el limite lo pone la maquina, no el codigo.
No habia ningun tope: el usuario N simplemente empujaba la caja a swap y
la sesion de TODOS se volvia lenta, que es peor que decirle al usuario N
que espere un minuto.
VELXIO_PI_MAX_INSTANCES (6) guests simultaneos en la maquina
VELXIO_PI_MAX_PER_OWNER (2) guests por persona
El "owner" es el hash de la cookie de sesion: sirve solo para contar, no
se guarda ni se lee de vuelta, y cae al host del cliente cuando no hay
cookie (sidecar de escritorio, tests). Sin identidad solo aplica el tope
global.
El rechazo llega como un mensaje que dice que hacer ("prueba en un
minuto" / "para una de tus sesiones"), no como un fallo mudo, y se
comprueba ANTES de lanzar el proceso.
Auditoria de "la Pi tiene todo lo de la placa real" con cuatro huecos
encontrados y cerrados:
1) GPIO de entrada en modo Linux: GPIO_IN respondia VAL 0 fijo (stub de
la fase 2), asi que GPIO.input() leia 0 eternamente aunque el canvas
empujara el nivel. El backend guarda ahora el ultimo nivel por pin
(set_pin_state lo escribe) y GPIO_IN contesta de ahi. Los flancos
(SET) siguen llegando al guest como antes.
2) UART del header hacia otra placa: el shim del rootfs ya hablaba
`UART <port> TX <hex>` / RX_REQ, pero sin modelo de esclavo el
backend tragaba los bytes. Ahora TX sin esclavo se emite al canvas
(uart_tx) y RX_REQ sin esclavo drena la cola que llena pi_uart_rx —
el mismo protocolo de siempre, sin ops nuevas.
3) El escaner de esclavos I2C/SPI/UART estaba doblemente muerto:
clasificaba por numero fisico de pin ('3','5','19'...) cuando el
elemento expone GPIOxx, y su unico llamador era RaspberryPiWorkspace,
que el terminal unificado reemplazo. Acepta ambos nombres y corre en
onBooted del store.
4) boardPinToNumber solo mapeaba los pines de la 3/4/5; la Zero, 1B+ y
2B (mismo header de 40 pines, mismo elemento) se quedaban sin mapa.
La placa QEMU-Linux tiene DOS flujos serie y hasta ahora el cableado
usaba el equivocado: el enrutado entregaba los bytes del vecino a la
consola (el shell) y sacaba al cable la cháchara del arranque. El
header, que es lo que el usuario cablea, no existia.
Ahora el canal de protocolo lleva dos ops nuevas:
UARTTX <b64> el guest transmitio por el header -> al canvas
UARTRX el guest pregunta que le llego -> UART_RXQ <b64>
El backend guarda una cola por instancia (acotada a 64 KB, que un script
que no lee nunca no la haga crecer) y el websocket acepta `pi_uart_rx`
con los bytes que el vecino manda. En el frontend el bridge gana
onUartTx / sendUartBytes y el Interconnect engancha ESE flujo en vez de
la consola para las placas Pi.
Con esto el mismo script -- import serial, escribir, dormir, leer --
funciona en los dos motores.
Guests stay on '-nic none' unless a profile opts in, and opting in does
NOT mean internet: the NIC is user,restrict=on (no route out, no route
to the host LAN) with one guestfwd to whatever command the overlay
configures — a filtering proxy in practice. Keeps 'user code never gets
a raw socket outside' true by construction.
A forgotten tab pinned a QEMU process and its RAM for as long as the
browser stayed open. Instances now shut themselves down after
MAX_SESSION_SECONDS (2 h default, env-tunable) and say so on the serial
line instead of vanishing.
A profile's extra_drive is the same file for everyone; an overlay may
need a disk built for THIS session (what the project declared). New
seam set_pi_extra_drive_resolver(fn) receives the client id, the board
and the start_pi payload and returns raw images, mounted read-only after
the profile's own. The WS route forwards msg_data and the bridge gained
startPayload so a client can declare it. Generic: no package manager,
no OS knowledge in the OSS tree.
- profile key extra_drive: optional read-only second virtio-blk so an
overlay can ship guest-side shim libraries (/dev/vdb)
- SENS <name> protocol op: canvas-fed named values (built-in sensors /
buttons) served from PiInstance.sensor_state, pushed by the frontend
via the new pi_sensor_state WS message
- DISP <b64> protocol op: guest display commands forwarded to the
frontend as 'display' events (built-in screens)
- RaspberryPi3Bridge: onDisplay / onGpioPwm callbacks + setSensorState
- SimulatorCanvas hands piFamily boards their Pi bridge in
attachBuiltins (was ESP32-only)
add_reader on the proto FIFO armed epoll but never delivered callbacks
under uvloop when the fd number recycled a just-closed socket fd
(nondeterministic per instance): the guest's GPIO lines sat unread in
the pipe and canvas LEDs stayed dark while serial kept flowing. Pump
the FIFO from a worker thread (select + os.read, like _watch_stderr)
and schedule _handle_gpio_line back onto the loop.
registerPiFamilyKind + ProBoardDef.piFamily route an overlay board through
the existing Raspberry Pi bridge path (WebSocket qemu, VFS panel, boot
terminal); register_pi_board_profile lets the overlay add the matching
PI_CONFIGS entry (cpu/image_set/...) at registration time.
Un crash entre ambos (el NameError de board_fqbn) dejaba defaults==rendered
con un sdkconfig stale, y el re-seed de kconfig no disparaba nunca mas en ese
dir persistente — medido: CONSOLE_SECONDARY quedo en usb_serial_jtag pese a
los defaults nuevos. El bloque de velxio_board.cmake pasa a despues.
El bloque de velxio_board.cmake usaba board_fqbn, que vivia en compile() y no
en la firma de _compile_in_dir: parametro nuevo con default None, propagado en
los dos call sites, y guard para el camino sin fqbn (tests directos).
Tres piezas:
- _arduino_board_flags lee <id>.build.board y <id>.build.cdc_on_boot de
boards.txt (cache, mismo patron que _arduino_variant).
- Cada compile escribe velxio_board.cmake (incluido OPTIONAL por el
CMakeLists ANTES de project(), asi los defines llegan a TODOS los
componentes — que Serial sea el CDC se decide dentro del core Arduino):
ARDUINO_<BOARD> siempre que boards.txt lo nombre (los sketches de vendor
hacen #ifdef ARDUINO_XIAO_ESP32S3), y ARDUINO_USB_CDC_ON_BOOT=1 +
ARDUINO_USB_MODE=1 cuando cdc_on_boot=1. USB_MODE va FORZADO a 1 (HWCDC
por USB-Serial-JTAG, la opcion de menu USBMode=hwcdc): el motor modela esa
consola, no la pila OTG/TinyUSB. Ir por archivo y no por env hace el flag
dependencia de configure: CMake reconfigura al cambiar.
- CONFIG_ESP_CONSOLE_SECONDARY_NONE en el sdkconfig: sin ella el ROM/IDF
espejan cada byte de log al USB y el monitor unico del emulador (que
mezcla ambos streams) mostraria el log entero dos veces en builds CDC.
_normalize_options rellena arduinoVariant con _arduino_variant(board_fqbn),
que ya leia boards.txt; la plantilla ya consumia la clave. Sin esto todas las
placas S3 compilaban como esp32s3 generico: sin nombres Dx del XIAO y con
Serial en UART0, cuyo RX por defecto es GPIO44 = D7 del XIAO — un
pinMode(D7) tardio desinstala el driver y silencia los prints del loop
(medido en el banco con el ejemplo del Round Display).
Sin esta linea la variante se queda en el nombre del chip, asi que TODA placa
compila como dev-kit generico y variants/<VARIANTE>/pins_arduino.h nunca entra
en el include path: los nombres propios de la placa (D0..D10 en una XIAO, la
serigrafia de las M5) sencillamente no existen. Justo los sketches que la gente
copia de la wiki del fabricante.
El helper que deriva la variante del FQBN leyendo boards.txt ya estaba; faltaba
la plantilla que la consume. Sigue pendiente rellenar 'arduinoVariant' en
normalized para que surta efecto (ver F7.6 en velxio-prod).
Los dos ejemplos de camara (esp32cam-lcd-preview y esp32cam-webcam-demo) morian
con "esp_camera.h: No such file or directory".
No es que el ejemplo estuviera mal: es una regresion de la migracion. El arbol
viejo /opt/arduino-esp32 (2.x) traia un SDK precompilado con esp32-camera dentro,
en tools/sdk/<target>/include/esp32-camera/, para esp32, c3, s2 y s3. El
componente 3.x que usamos ahora NO lo trae, y su idf_component.yml tampoco
depende de el, asi que el header simplemente dejo de existir. Por eso funcionaba
antes y dejo de funcionar sin que nadie tocara el ejemplo.
Mezclar los headers de la 2.x en este build no es opcion — vienen con su SDK
precompilado y aqui compilamos contra IDF 5. Lo correcto en el mundo 3.x es
pedirle el componente al gestor: la plantilla no traia idf_component.yml, asi que
todo lo gestionado llegaba de rebote por el manifiesto de arduino-esp32, y lo que
el no arrastra hay que pedirlo explicitamente.
_detect_camera_usage() ve el include y _add_managed_components() escribe
main/idf_component.yml con espressif/esp32-camera. Verificado en el contenedor
sobre el mismo directorio que fallaba: el gestor descarga 2.1.7, el configure
termina con esp32-camera en la lista de componentes, y sketch.ino.cpp compila.
Arduino trata igual #include "Lib.h" que #include <Lib.h> para librerias, y
los ejemplos de los propios fabricantes usan la forma con comillas: los de
M5Stack para el Cardputer abren con #include "M5Cardputer.h". Al escanear solo
la forma con angulos, ese sketch no llegaba siquiera al resolutor de librerias y
moria con 'fatal error: M5Cardputer.h: No such file', mientras el mismo sketch
con angulos compilaba sin problema. Cualquiera que pegue un ejemplo oficial se
daba de bruces con esto.
Se pasan ademas los nombres de los ficheros del propio sketch, para que un
#include "Common.h" local siga siendo una cabecera del proyecto y no se
confunda con una libreria.
test/backend/unit/test_espidf_compiler.py cubre ambas formas, el espaciado, las
cabeceras internas de IDF y la cabecera propia del proyecto.
Las dos ramas habian divergido: master llevaba el modo lenguaje ESP-IDF puro
(#139) y v3.2 la ruta de compilacion IDF v5.5 para toda la familia ESP32 mas
los arreglos de venv/toolchain. Ambas tocaban espidf_compiler.py.
Los dos lados son ejes ORTOGONALES y se conservan enteros:
- use_idf5 / arduino_mode (v3.2): que arbol IDF usa el build (5.5 vs 4.4) y
si cabe Arduino-como-componente.
- pure_idf (master): el modo LENGUAJE que elige el usuario; sus ficheros son
las fuentes del componente main con su propio app_main().
Resolucion:
- _build_env acepta los tres. Un build IDF puro fuerza arduino_mode a falso:
la plantilla CMake mete el componente arduino-esp32 en cuanto existe
ARDUINO_ESP32_PATH, asi que dejarlo puesto compilaba el core de Arduino en
un build que no tiene sketch. Lo cazaron los tests de master.
- VELXIO_PURE_SKETCH solo con pure_idf, nunca con arduino_mode a falso a
secas: un target sin core arduino-esp32 sigue entregando un SKETCH al
traductor legacy y no debe tomar la rama del glob puro.
- La identidad del build-dir suma los dos tokens (|idf:N|ard:N y |lang:pure):
ningun par de esas combinaciones puede compartir un build/ configurado.
- La cadena de escritura de fuentes queda pure_idf -> arduino_mode -> legacy.
- sdkconfig: render de v3.2 (con target/use_idf5) mas el filtrado de simbolos
CONFIG_ARDUINO* de master cuando el build es puro.
test/backend/unit/test_espidf_compiler.py: los 7 tests que ya estaban rotos en
v3.2 (AttributeError: idf5_path, fixture sin actualizar desde que se anadio la
seleccion de IDF) vuelven a pasar.
Verificado: backend 293 pasan / 0 fallan (v3.2 traia 7 rotos); frontend 2268
pasan / 0 fallan en los dos shards.
Adds a third entry to the board language selector next to Arduino C++
and MicroPython: ESP-IDF. In this mode the user writes a plain ESP-IDF
project — app_main() entry point, FreeRTOS + driver APIs — and the
backend compiles it through the same ESP-IDF toolchain it already uses
for ESP32 Arduino sketches, just without the arduino-esp32 component.
Backend:
- CompileRequest.language ('espidf') threaded through the sync + async
compile paths and folded into the dedup job key (language='arduino'
and omitted hash identically so old clients keep dedupping).
- espidf_compiler: pure_idf flag. User files are written into main/
as-is (no Arduino.h wrap, no velxio_compat.h, Arduino library
resolution skipped), ARDUINO_ESP32_PATH is dropped from the build env
and VELXIO_PURE_SKETCH raised so the template CMake compiles the
user's own sources via a glob branch. Pure builds get their own
persistent build-dir variant through the eff_hash fold.
- QEMU WiFi compat for IDF-style code: esp_wifi.h/esp_wifi_init
detection sets has_wifi, and literal #define SSID/PASS plus
wifi_config_t designated initializers are normalized to the QEMU AP.
- CONFIG_ARDUINO_* lines are stripped from sdkconfig.defaults in pure
mode (the symbols don't exist without the arduino component).
Frontend:
- LanguageMode gains 'espidf'; BOARD_SUPPORTS_ESPIDF covers the ESP32
family (Xtensa, S3, C3). Toolbar shows the option only for those.
- Switching modes seeds a main.c blink skeleton (app_main + gpio
driver), mirroring the MicroPython main.py flow.
- compileCode sends language='espidf'; run/stop paths are unchanged
(the QEMU worker consumes the same merged flash image).
- New gallery example: esp32-idf-blink (LED + resistor on GPIO 2).
Tests: unit coverage for the build-env switch, IDF wifi normalization,
job-key variance, file-group seeding and the new example; verified
end-to-end in a container from the prod image (pure build produces a
bootable flash image; Arduino-mode build unchanged, same variant hash).
Ports the v5 compiler layer: /opt/esp-idf-v5 + arduino-esp32 3.x
discovery (IDF5_PATH / ARDUINO_ESP32_5_PATH overridable), per-compile IDF
selection routing the family to the v5 tree, esp32c6 target support, and
the docker-runtime hardening (version-matched python venv pinned on PATH,
toolchain ordering by the tree's tools.json, user-libs component REQUIRES
for the IDF components Arduino libraries include directly). The LED_BUILTIN
fallback in the project template becomes a whitelist so non-esp32 variants
(C6 first) never hit the redefinition trap. Includes the option tests.
The newest-env-wins heuristic pinned a 5.x env for a 4.4 tree — each
generation's dependency check only accepts its own requirement set. The
version is read from tools/cmake/version.cmake of the active IDF tree,
falling back to the newest env when unreadable.
Two failure modes seen on a fresh docker deploy where the final image's
system python differs from the one the IDF venv was created with:
- IDF's cmake derives the venv dir name from the SYSTEM python version
and fails with 'python doesn't exist' before configure; and when cmake
is invoked directly (not via idf.py) its PYTHON property falls back to
the bare 'python' from PATH. The Linux branch of _build_env now pins
IDF_PYTHON_ENV_PATH to the newest env under the tools root and puts its
bin first on PATH (mirroring what the Windows branch already did).
- When several toolchain generations share one tools root, glob order
decided which won the PATH race. Toolchain dirs are now ordered by the
versions the IDF tree declares in tools/tools.json.
_emit() wrote to stdout synchronously from QEMU callback context — the
iothread fires _on_uart_tx per UART byte, each becoming ~45 bytes of
JSON. When the parent's pipe reader stalled, the 64 KB pipe filled and
the write blocked INSIDE the QEMU iothread, freezing the entire guest.
Observed as an intermittent (~1 in 10) boot hang: serial output stops
right after the ROM log / rtcinit line — exactly where the accumulated
boot events cross the pipe capacity — and never recovers. Direct worker
harness runs (fast reader) never reproduced it; the browser/WS path did.
Route _emit() through a bounded queue drained by a dedicated writer
thread (opportunistic batching, one write per drain). Under extreme
backpressure events are dropped and counted on stderr — losing telemetry
beats freezing the emulated CPU. The shutdown path posts a sentinel and
joins the writer so crash/system events still flush before os._exit.
Verified on staging: 10/10 UI run/stop cycles + 8/8 direct harness boots
with identical serial latency (~0.95s to first app output).
_refresh_pin_pulls read a fixed 40 registers from get_internals(3), but
the exposed array is per-chip: classic muxgpios[40], ESP32-S3 49
(GPIO0..48). The fixed bound silently missed S3 pulls on GPIO40-48.
Read _GPIO_COUNT entries instead (set from the machine at startup).
Pairs with qemu-lcgamboa 547e989, which models the S3 IO_MUX pull bits
and exposes them via the get_internals override — together they make
INPUT_PULLUP buttons work on the emulated S3 exactly like the classic
ESP32 (verified on staging with GPIO4 and GPIO40).
The S3 has GPIO0..48; the identity pinmap length drives how many pins
picsimlab_wire_gpio connects to the host. Default 40 left GPIO40..48 unwired,
so digitalWrite/Read on those pins never reached the frontend. Build a 49-entry
pinmap when the machine is esp32s3 (mirrors the c3->22 special case). Requires
the GPIO-model widening in libqemu (esp32_gpio bank-1 8->17).
_idf_target had no S3 case, so every S3 FQBN (esp32s3 / XIAO_ESP32S3 /
nano_nora) compiled as plain esp32 (LX6) and could not boot the S3 QEMU
machine. Add _is_esp32s3 (covers all three FQBNs; nano_nora has no 's3'
token) + return 'esp32s3'; add the xtensa-esp32s3-elf toolchain to the
build PATH; place the S3 second-stage bootloader at flash offset 0x0
(like C3) instead of 0x1000 via 'is_c3 or idf_target==esp32s3' at the merge.
Requires the xtensa-esp32s3-elf toolchain in the image
(install.sh esp32,esp32c3,esp32s3) and a libqemu-xtensa with an
esp32s3(-picsimlab) machine.
Extend the spice-driven input path (already live for AVR/ESP32) to RP2040 and
STM32 so digitalRead() of an INPUT pin reflects the actual wiring: a pin tied
to a rail reads that rail, and an INPUT_PULLUP button-to-GND reads idle-HIGH /
pressed-LOW instead of floating or inverted.
RP2040 (rp2040js, frontend-only): the GPIO listener now splits input vs output
mode. Input pins report their pad pull (InputPullUp/Down) via setPinPull and
seed the pull's idle level (rp2040js does not auto-apply the pad pull to the
readable input register); the SPICE solve then overrides via connectDigital-
InputsToMcu when the net is actually sourced. Output pins drive as before.
spiceDrivenInputs = true.
STM32 (backend QEMU): the worker now forwards a new gpio_pull event (from the
libqemu-arm picsimlab_pull_pin callback) so the netlist stamps the matching
weak resistor; Stm32Bridge surfaces it, Stm32BridgeShim opts into
spiceDrivenInputs, and collectPinStates maps PA0/PC13 names to the linear pin
so the pull is read. STM32 outputs stay on the part layer (unchanged).
Event-driven parts with no SPICE model (rotary encoder, keypad) remain
protected by the existing sourcedNets gate in the connector.
Completes the internal-pull emulation for the common case (a button on an
RTC-capable GPIO like 4/15/25/... with INPUT_PULLUP):
- Backend reads the RTC_IO pad RUE/RDE bits via the new
get_internals(QEMU_INTERNAL_RTCIO) and emits gpio_pull for RTC pins, so
pull-up/down on those pads is finally visible (it lives in RTC_IO, not
IO_MUX). IO_MUX path still covers non-RTC pins.
- The digitalRead path is driven by seeding the GPIO input level, not by
SPICE. The part-level INPUT_PULLUP seed (BasicParts) is sent at attach,
before the multi-second QEMU boot finishes, so it is lost and the pin
reads LOW. makePinPullHandler now drives the pin to the pull's idle level
via sendPinEvent when the guest programs the pull (post-boot), so it
sticks. A real button press/release still overrides it.
INPUT_PULLUP / INPUT_PULLDOWN had no effect in simulation: the ESP32's
internal pull resistors live inside QEMU and were invisible to the SPICE
solver, so an input wired to a button-to-GND floated to 0 V and read LOW
even at idle. The canonical active-low button never worked.
Read the pull config straight out of the running guest: the IO_MUX
register (FUN_PU bit 8 / FUN_PD bit 7) is already exposed read-only via
qemu_picsimlab_get_internals(3), so no QEMU rebuild is needed. The worker
scans it on the 100 ms poll thread and emits gpio_pull; the bridge feeds
it to PinManager; the netlist stamps a weak 45k resistor to the rail so
idle inputs read the correct level. 45k matches the real internal pull
and is weak enough that any external driver/pull dominates.
Verified with ngspice: idle ~3.3 V (HIGH), pressed ~0 V (LOW).
David Montero Crespo