A MicroPython library in Velxio IS a .py file in the project, copied onto
the board's filesystem before main.py runs. That mechanism worked, but
getting the file meant hunting GitHub by hand — the gap issue #214 is
about. Now the Library Manager fills it, in both halves:
Backend: /api/micropython-libs proxies the official micropython-lib
package index (the same one `mip` uses on real hardware). /search filters
the index; /fetch resolves a package plus its declared deps into .py
files. The proxy is what makes this safe to expose: only the official
index is reachable (the name must exist in index.json, files come from its
content-addressed store), source only (.py, never .mpy bytecode), path-
safe, and capped at 24 files / 256 KB per install. In-process TTL cache
over the index.
Frontend: the same Library Manager modal, mode-aware. On a MicroPython
board the search hits the index and "Add to project" writes the package's
files into the board's workspace (upsert — re-adding is the upgrade path),
visible in the explorer and travelling with the project. Before typing it
shows featured picks (ssd1306, umqtt.*, urequests) and the modules already
frozen into the firmware (neopixel, dht, onewire, ds18x20, machine,
network) labeled "built into the firmware", so searching for one answers
"just import it" instead of an empty list.
The file-materialization prelude now creates package directories: umqtt
ships as umqtt/simple.py, and MicroPython's open() does not mkdir, so the
prelude died with ENOENT on any packaged module.
Also ships a gallery example ("ESP32 + MicroPython: use an external
library") that documents the file-is-the-library mechanism with the
ssd1306 driver on the wired OLED.
arduino-cli reports errors as plain text or, with --format json, as a JSON
object on stderr. We forwarded that blob verbatim, so on a fresh install
with no library index yet (and no way through the firewall/proxy to
downloads.arduino.cc) the in-app agent told the user "the library server
returns a JSON error" ten times in a row.
humanize_cli_error() maps the known shapes — index download failed,
library zip download failed, library not found — to a single sentence
that says what to check, and keeps the raw text under raw_error. Applied
to search_libraries (rc != 0 and non-JSON stdout) and install_library.
Tests use the real arduino-cli 1.0.4 output captured offline.
ESP-IDF compiles with -ggdb, and for debug builds ccache folds the cwd into
every cache key by default — CCACHE_BASEDIR does not exempt it (verified with
CCACHE_DEBUG on 4.11: 'Hashing CWD /var/lib/velxio-build/esp32s3/v_<hash>/
project/build'). Each build variant lives in its own dir, so every variant had
a private key space: eight esp32 variants of the SAME board each recompiled
all ~2100 IDF objects from scratch, the cache filled with dead per-directory
entries, and a cold Cardputer/S3 build (2263 steps) overran the 600s ninja
timeout on a busy box. CCACHE_NOHASHDIR=1 makes variants of the same
board/sdkconfig share objects; a cold variant becomes mostly cache hits. Only
DW_AT_comp_dir in .debug_info of a reused object can differ, which never
reaches the flashed .bin.
The timeout paths were silent server-side; they now log the build dir and how
many ninja steps completed, and the user-facing message says the partial
build is kept and a retry resumes.
Measured against the user corpus, the by-resolved-type long tail was
losing what the old catch-all happened to include: for `Adafruit_SH1106G
display(...); display.setTextSize(` the class itself declares one method
(begin) and everything people call lives four levels up (Adafruit_SH110X ->
Adafruit_GrayOLED -> Adafruit_GFX -> Print). Same story for MCUFRIEND_kbv,
Adafruit_NeoMatrix and every GFX-derived display driver.
Three changes:
- The scanner records `bases` on class symbols (access/virtual keywords
dropped, `ns::Base` -> Base, template args cut). 2,529 classes across the
production cache now carry their bases.
- /type/{name} walks the chain breadth-first through the type index (bases
routinely live in a different library), re-owns inherited members to the
requested type so the editor sees one flat class, tags them with
`inheritedFrom`, and lists the chain in `bases` -- including bases NOT in
the cache (Print, Stream, or an embedded one like Adafruit_GFX) so the
host can supply those itself. Depth capped at 8. Adafruit_SH1106G goes
from 1 member to 64.
- `void a(int), b(), c(char);` declares three methods; the extractor took
only the first. TFT_eSPI declares setTextColor/setTextSize and
setCursor/getCursorX/... this way. Split on top-level commas after a
complete `name(args)` group. TFT_eSPI 118 -> 173 members.
Rescan of the 1,125-entry production cache: +274 member symbols, 15
libraries gain, none lose.
The completion-safe long-tail path. Feeding whole library symbol sets into
the editor's candidate pool was measured to make ranking worse (U8g2 alone
injects ~250 module-level class names; ArduinoJson dozens of ARDUINOJSON_*
macros). What does help is answering `client.` once tree-sitter has resolved
`client` to a declared type the embedded catalog does not know -- and for
that the engine needs "the members of type T", not "the symbols of library
L". This endpoint returns exactly that: symbols whose owner is the requested
class, from whichever cached library declares it, in the same payload shape
as /symbols/ so the engine's RemoteSymbolCache stores it unchanged.
Resolution: a class-name -> cache-entry index built once over the library
cache (each per-entry scan is already disk-cached), persisted under
.symbols/_types.json (3,818 classes / 199 KB over the 1,125-entry
production cache; ~1 s to rebuild from cached scans, ~60 s cold), refreshed
after a few hours or when the cache grows. Many classes are declared by more
than one library -- the real one plus forks and bundles vendoring a copy
(PubSubClient inside "ESP8266 Microgear", NTPClient inside "ESP8266 Weather
Station", AccelStepper inside "Arduino Learning Board") -- so the pick is:
the library named after the class, else the one declaring the most members
of it (vendored copies are trimmed), else alphabetical.
Generic, not pro-specific: any host with a library cache can use it.
_scan_text applied `public:` / `private:` / `protected:` only inside
handle_statement, which the `{` branch skipped whenever the statement opened
a nested enum, struct or class. So `public:\n typedef enum { ... }` -- the
first thing in AccelStepper's class body, and a common idiom -- took the enum
branch, the label was never seen, the class stayed in its default private
access, and every method after the enum was dropped. AccelStepper shipped 4
of its public methods; DallasTemperature, SoftWire, MFRC522_I2C, RTCZero
shipped none.
Labels are now consumed before the branch. Rescanned all 1,125 entries of
the production library cache: 82 libraries gain members (+1,532 total,
DallasTemperature 0 -> 57), and the 20 that "lose" one or two were leaking
private/protected members past the old check (PMS::loop is under private:,
hd44780::waitReady under protected:) -- precision, not regression.
`_prepare_persistent_project_dir` keys the persistent build dir on
(idf_target, variant), but the lock was keyed on the FQBN. Several FQBNs
resolve to one variant — arduino-esp32's boards.txt gives both `esp32` and
`esp32cam` build.variant=esp32 — so those two were free to build in the SAME
directory at the same time.
Measured against staging, compiling both concurrently:
esp32 285504 bytes sha f65ca909… contains its own marker
esp32cam 285504 bytes sha f65ca909… contains the OTHER sketch's marker
Byte-identical firmware, and the esp32cam caller got the esp32 sketch's
binary. In the product that is two gallery tabs open at once: the first tab
prints the second example's serial log and does nothing itself. It made
esp32-pwm-led-rgb look broken (dark LED, every SPICE node 0 V) when the
example is fine — it was running someone else's program.
Key the lock on the same identity as the directory. Boards with genuinely
separate build dirs still compile in parallel up to the semaphore cap; an
unresolvable FQBN falls back to the default pair, which is the safe
direction (serialising more only costs throughput).
Also in this commit: battery-voltage-monitor ships the 9 V battery its own
description promises. The divider's top end was wired to nothing, so A0 read
0 V and the sketch printed "Battery: 0.0 V / WARNING: Battery low!" forever.
The custom sketch.ino.cpp path never ran arduino-cli's ctags prototype
pass, so the universal Arduino idiom (helpers defined after loop()) died
with "'foo' was not declared in this scope" on every ESP32 build. A
conservative generator (skips templates, default args, methods,
function-pointer params) injects prototypes before the first definition
with a #line directive so error line numbers stay put. 8 unit tests.
IDF 5.5 renamed ESP_TASK_WDT to the EN/INIT pair; the v4.4 template's =n
became EN=n, compiling the whole implementation out — every sketch using
esp_task_wdt_add/reset (Firebase's client does internally) failed at
link. The v5 fixup now emits EN=y INIT=n: the API links, the watchdog
never auto-starts under emulation. v4.4 semantics untouched.
GET /api/intellisense/symbols/{Name@Version} scans the content-addressed
library cache entry's headers (dependency-free C++ scanner) into an editor
symbol table, disk-cached per entry and served with long cache headers.
Plain OSS hosts without a lib cache 404 and the editor degrades gracefully.
C3/S3/C6 default to CONFIG_BT_BLE_50_FEATURES_SUPPORTED with 4.2 OFF, so
arduino-esp32's Bluedroid BLE library (esp_ble_gap_config_adv_data_raw,
esp_ble_gap_start_advertising) died at link with undefined references —
esp32c3-ble-advertise never compiled. Enabling 42+50 together matches
arduino-esp32's own per-target sdkconfigs; on classic ESP32 these flags
are already the Kconfig defaults. Found by the nightly gallery smoke.
The dead-branch pruner marked any assumed-live branch as 'taken', so the
#else of an UNKNOWN condition was skipped. GxEPD2 guards its GFX base
class with '#if ENABLE_GxEPD2_GFX (default 0) ... #else #include
<Adafruit_GFX.h>' — the pruned #else silently dropped the Adafruit_GFX
dependency and every GxEPD2 e-paper example failed with 'fatal error:
Adafruit_GFX.h: No such file'. Found by the nightly gallery compile smoke.
'taken' is now set only by PROVABLY-true conditions (all identifiers in
the known-true set), keeping the 2026-08-03 ESPAsyncWebServer behaviour:
provably-dead branches still prune, provably-true branches still kill
their #else.
The ESP32 examples now default to the in-browser JS engines, whose lwIP
serves the sketch's HTTP server from the simulation tab itself. Opening
the IoT gateway in a new tab backgrounds that tab, the browser throttles
its timers, and the in-chip server can't answer before the proxy's
timeout — the exact failure the Pico W iframe path already solved.
- WifiStatus grows an inBrowser flag (set by the overlay's JS bridges);
the serial-monitor link and the WiFi badge now open the in-app iframe
for any board that carries it, not just pi-pico-w.
- /api/gateway polls ~1s before answering 404: the net bridge dials on
'got_ip', which races the sketch printing its URL, so a fast click
used to land before the bridge registered.
The main component had them; the merged user-libraries component did not, so
a warning INSIDE a vendor library was still fatal. M5Stack's MahonyAHRS.cpp
(fast inverse square root, a float read through a long*) fails under
-Werror=uninitialized, which made installing the vendor's own library kill
every build using it. Library code the user cannot edit must not stop their
build; the diagnostics still print as warnings.
The library's fast inverse square root reads a float through a long*, GCC
flags y as used uninitialized, and -Werror turns a vendor library the user
cannot edit into a failed build. Demoted to a warning like its siblings.
Two of M5Stack's own published Cardputer examples fail to build here and
compile fine in the Arduino IDE: SSHClient passes an int to a %lu, and
mic_wav_record has a local GCC thinks may be used uninitialized. ESP-IDF
promotes both categories to errors, which is right for IDF itself and wrong
for a sketch a user pasted from the vendor's repository — the emulator looks
broken for code that is not.
format and maybe-uninitialized join the existing relaxation list, which
already carries comment, parentheses, sign-compare and the rest for exactly
this reason. They stay warnings, so the diagnostic is still printed; -Wall is
untouched.
mcp 2.0.0 shipped on 2026-07-28 and deleted mcp.server.fastmcp, the module
app/mcp/server.py imports. requirements.txt asked for `mcp>=1.0.0` with no
ceiling, so the very next CI run resolved the new major and died importing
test_mcp_tools.py.
That failure is a COLLECTION error, not a test failure: pytest aborts with
exit 2 before running anything, so all 295 unit tests have been silently
unrun on every push since. The workflow was red, but red for a reason
nobody's code caused.
The deploy venv escaped only by accident — it was built while 1.x was still
current, and deploy.sh rebuilds it when requirements.txt changes, so the
next rebuild would have walked into the same wall and taken the deploy gate
down with it.
Verified in a clean venv built from this file: resolves to mcp 1.29.0,
322 passed / 18 skipped, test_mcp_tools.py's 35 tests among them.
Raising the cap means porting to the 2.x server API (mcp.server.mcpserver).
useProjectStore.currentExampleId — stamped by loadExample, dropped the
moment a real project (or a blank workspace) takes over. The compile
request and the run event carry it as example_id, analytics only, so
'which examples get compiled and run most' becomes answerable.
Several boards share one FQBN (the Pimoroni RP2350 trio all compile as
rpipico2), so FQBN-keyed metrics can't tell them apart. The compile
request and the run event now carry board_kind — analytics only, rides
into the metric hook's extra.
suppressOnlineOnlyAds(): overlay seam so the hosted build can silence
'available in the online editor' ad cards for items it manages — an
embargoed item must vanish, not advertise the site the user is on.
Pure OSS builds never call it; ads behave exactly as before.
Declared inside the has_arduino branch, so every successful pure
ESP-IDF (language=espidf) build crashed with UnboundLocalError when the
success return read it. Regression from today's manifest auto-migration
change; the Arduino paths were live-verified but the espidf mode was
not.
NewsMarkdown (shared renderer): YouTube links render as a thumbnail +
play badge that swaps to the youtube-nocookie iframe on click (nothing
loads from the player until the user opts in); images render inline;
all links open in a new tab. Product decision supersedes the alt-text
image stripping: posts embed screenshots/demos, and the README now
documents that embedded media loads in the viewer's browser.
Security review follow-ups (all LOW):
- proxy: negative cache on upstream failure (no outbound fetch per
request while velxio.dev is unreachable) and a 2MB body cap via
streaming read (httpx timeouts are per read op, not per body)
- NewsModal: OSS builds render markdown images as alt text so viewers'
browsers never contact remote hosts; pro cloud builds show them
Editor shows scheduled product-news posts as a markdown modal: max one
per day, oldest first, each post only ever once (marked seen at
delivery). OSS default source proxies velxio.dev's public feed through
the local backend (cached ~6h, anonymous, VELXIO_NEWS=off to disable)
with seen-state in localStorage; a pro overlay can register a
server-backed per-user source via lib/newsSource.ts.
File names may now carry '/' paths; the explorer renders each board group
as a collapsible folder tree (folders first, files in group order, CSS-var
indentation because .fe-file-item pins padding with !important). New-folder
buttons on the board header and per-folder hover actions (new file here,
new subfolder, delete with confirm when files are inside; a folder holding
every file of the group cannot be deleted - groups keep >= 1 file).
Model: names are the source of truth; only EMPTY folders are tracked
(folderGroups per group) so they survive until a file lands in them, then
materialise into the name prefix. Round-trips through .vlx (optional
folderGroups field, laxly validated), the auth stash draft, and
loadProjectState. Examples gain boards[].files so a single board can ship
a full multi-file tree (first entry = main sketch).
Backend: the arduino-cli sketch writer now creates subdirectories for
'/'-pathed names and REJECTS names that resolve outside the sketch dir
('..' traversal) - previously it crashed on the former and wrote outside
the tmp dir on the latter.
Three-part fix for the library-contamination class of ESP32 compile
failures (a user's project kept pulling unrelated libraries because 79%
of projects compile with an empty manifest, forcing scan-all resolution
over ~1000 shared libraries):
1. Name-affinity ranking in _find_library_for_header: the provider whose
name matches the header stem wins (exact normalized match > esp32
arch declaration > contains-stem, alphabetical only as tie-break).
Previously first-alphabetical won, so any stray library shipping a
generic header (Servo.h has 6 providers) could hijack the build.
2. Manifest auto-migration: scan-all compiles now report the libraries
the build really used (manifest_incomplete +
manifest_suggested_libraries on the compile result); the editor folds
single-candidate suggestions into the board's declared manifest after
a green build, so the NEXT compile runs scoped. One green build = one
migrated project; empty-manifest scan-all behaviour is untouched.
3. Install-declares: the Wokwi import library modal records successful
installs in the active board's manifest, and .vlx export/import now
round-trips board.libraries.
Tests: test_espidf_library_ranking.py (9) + library-manifest.test.ts (19).
Reported 2026-08-03: every ESP32 build in a user's project failed with
errors from libraries he never installed, emitted from user_libs_all --
a folder his file list does not even show, in projects he had recreated
from scratch.
Root cause: _detect_external_includes was a bare regex with no
preprocessor state. ESPAsyncWebServer guards <Hash.h> behind
'#if defined(ESP8266) || defined(TARGET_RP2040)', so the compiler never
sees it on ESP32 -- but the scanner reported it anyway. The resolver
then searched the ~1000 shared-cache libraries, matched stemi-hexapod's
src/Hash.h (a hexapod-robot library), merged it WHOLE, and compiled all
of its sources; each failed on its own unrelated includes.
The scanner now tracks #if/#ifdef/#ifndef/#elif/#else/#endif nesting and
strips comments. A branch is skipped ONLY when every identifier in its
condition is a platform macro known to be undefined on ESP32; unknown
macros keep the branch LIVE, because hiding a genuinely needed header
would be a much worse failure than the old over-reporting. Verified
against the real ESPAsyncWebServer shape plus nesting/elif/else/comment
cases (9 new tests, 15 green in the espidf suites).
Complements the platform-header guard from the previous commit, which
stays as defence in depth.
Reported by a user whose every ESP32 build failed with errors from
libraries he never installed (SoftwareSerial.h, ArduinoBearSSL.h,
Adafruit_GFX.h, AsyncElegantOTA.h) coming out of user_libs_all, in a
project whose file list doesn't even show that folder.
Traced end to end: ESPAsyncWebServer includes <Hash.h> (an ESP8266-core
header, guarded for that platform, so arduino-esp32's tree has no such
file). The global scan then resolved it to stemi-hexapod's src/Hash.h --
a hexapod-robot library -- merged the WHOLE library into user_libs_all,
and its Serial.cpp / Hash.cpp / Server.cpp / BluetoothLowEnergy.cpp /
ExpansionDriver.cpp were compiled, each failing on its own unrelated
includes. Log line: 'Merging "stemihexapod" into user_libs_all for
<Hash.h>'.
Hash.h and friends now join the never-resolve-against-user-libs set.
They cannot be discovered by the core-tree scan (the core genuinely
lacks them) yet their names are generic enough that some installed
library owns a file by that name.
Installing ArduinoJson made the project stop compiling, with an error that
never mentioned ArduinoJson:
CMake Error at user_libs/user_libs_all/CMakeLists.txt:15
(target_compile_definitions):
target_compile_definitions may only set INTERFACE properties on
INTERFACE targets
-- Configuring incomplete, errors occurred!
ArduinoJson ships headers only. With no sources, idf_component_register()
builds the merged user-libs component as an INTERFACE library, and PRIVATE
scope on an INTERFACE target is a hard CMake error - so the configure step
died before a single file was compiled. The user saw only a failing build
and reported it as "ArduinoJson can't be found even after it has been
installed" (#124).
Those definitions map alloca/memcpy_P/memcmp_P for library C sources that
use them undeclared; with no sources there is nothing to define, so they are
emitted only when the component actually compiles something. A library that
carries its own .cpp is unchanged - which is why LiquidCrystal_I2C (#257)
worked and hid this.
Two-tier rule replacing the blanket skip:
- 'all' now counts as wildcard ('*'): LiquidCrystal_I2C 2.0.0 declares
architectures=all and was skipped on every ESP32 build, breaking the
most popular I2C LCD library with a bare 'No such file or directory'.
- Headers the SKETCH includes directly merge even when the library
declares a foreign-only architecture (many avr-declared libs are pure
Wire/SPI code that compiles fine; a truthful compile error beats a
missing-header one). Transitive pulls keep the HARD guard, which is
the path that dragged SAMD-only Adafruit_ZeroDMA into ESP32 builds.
Lets metrics overlays distinguish agent-triggered compiles from manual
ones (the AI assistant's compiles were previously invisible in project
counters). Loose optional field — omitted means manual.
Cada guest es un proceso QEMU con su propia RAM (1-2 GB segun placa) y
sus hilos de vCPU, asi que el limite lo pone la maquina, no el codigo.
No habia ningun tope: el usuario N simplemente empujaba la caja a swap y
la sesion de TODOS se volvia lenta, que es peor que decirle al usuario N
que espere un minuto.
VELXIO_PI_MAX_INSTANCES (6) guests simultaneos en la maquina
VELXIO_PI_MAX_PER_OWNER (2) guests por persona
El "owner" es el hash de la cookie de sesion: sirve solo para contar, no
se guarda ni se lee de vuelta, y cae al host del cliente cuando no hay
cookie (sidecar de escritorio, tests). Sin identidad solo aplica el tope
global.
El rechazo llega como un mensaje que dice que hacer ("prueba en un
minuto" / "para una de tus sesiones"), no como un fallo mudo, y se
comprueba ANTES de lanzar el proceso.
Auditoria de "la Pi tiene todo lo de la placa real" con cuatro huecos
encontrados y cerrados:
1) GPIO de entrada en modo Linux: GPIO_IN respondia VAL 0 fijo (stub de
la fase 2), asi que GPIO.input() leia 0 eternamente aunque el canvas
empujara el nivel. El backend guarda ahora el ultimo nivel por pin
(set_pin_state lo escribe) y GPIO_IN contesta de ahi. Los flancos
(SET) siguen llegando al guest como antes.
2) UART del header hacia otra placa: el shim del rootfs ya hablaba
`UART <port> TX <hex>` / RX_REQ, pero sin modelo de esclavo el
backend tragaba los bytes. Ahora TX sin esclavo se emite al canvas
(uart_tx) y RX_REQ sin esclavo drena la cola que llena pi_uart_rx —
el mismo protocolo de siempre, sin ops nuevas.
3) El escaner de esclavos I2C/SPI/UART estaba doblemente muerto:
clasificaba por numero fisico de pin ('3','5','19'...) cuando el
elemento expone GPIOxx, y su unico llamador era RaspberryPiWorkspace,
que el terminal unificado reemplazo. Acepta ambos nombres y corre en
onBooted del store.
4) boardPinToNumber solo mapeaba los pines de la 3/4/5; la Zero, 1B+ y
2B (mismo header de 40 pines, mismo elemento) se quedaban sin mapa.
La placa QEMU-Linux tiene DOS flujos serie y hasta ahora el cableado
usaba el equivocado: el enrutado entregaba los bytes del vecino a la
consola (el shell) y sacaba al cable la cháchara del arranque. El
header, que es lo que el usuario cablea, no existia.
Ahora el canal de protocolo lleva dos ops nuevas:
UARTTX <b64> el guest transmitio por el header -> al canvas
UARTRX el guest pregunta que le llego -> UART_RXQ <b64>
El backend guarda una cola por instancia (acotada a 64 KB, que un script
que no lee nunca no la haga crecer) y el websocket acepta `pi_uart_rx`
con los bytes que el vecino manda. En el frontend el bridge gana
onUartTx / sendUartBytes y el Interconnect engancha ESE flujo en vez de
la consola para las placas Pi.
Con esto el mismo script -- import serial, escribir, dormir, leer --
funciona en los dos motores.
Guests stay on '-nic none' unless a profile opts in, and opting in does
NOT mean internet: the NIC is user,restrict=on (no route out, no route
to the host LAN) with one guestfwd to whatever command the overlay
configures — a filtering proxy in practice. Keeps 'user code never gets
a raw socket outside' true by construction.
A forgotten tab pinned a QEMU process and its RAM for as long as the
browser stayed open. Instances now shut themselves down after
MAX_SESSION_SECONDS (2 h default, env-tunable) and say so on the serial
line instead of vanishing.
A profile's extra_drive is the same file for everyone; an overlay may
need a disk built for THIS session (what the project declared). New
seam set_pi_extra_drive_resolver(fn) receives the client id, the board
and the start_pi payload and returns raw images, mounted read-only after
the profile's own. The WS route forwards msg_data and the bridge gained
startPayload so a client can declare it. Generic: no package manager,
no OS knowledge in the OSS tree.
- profile key extra_drive: optional read-only second virtio-blk so an
overlay can ship guest-side shim libraries (/dev/vdb)
- SENS <name> protocol op: canvas-fed named values (built-in sensors /
buttons) served from PiInstance.sensor_state, pushed by the frontend
via the new pi_sensor_state WS message
- DISP <b64> protocol op: guest display commands forwarded to the
frontend as 'display' events (built-in screens)
- RaspberryPi3Bridge: onDisplay / onGpioPwm callbacks + setSensorState
- SimulatorCanvas hands piFamily boards their Pi bridge in
attachBuiltins (was ESP32-only)
add_reader on the proto FIFO armed epoll but never delivered callbacks
under uvloop when the fd number recycled a just-closed socket fd
(nondeterministic per instance): the guest's GPIO lines sat unread in
the pipe and canvas LEDs stayed dark while serial kept flowing. Pump
the FIFO from a worker thread (select + os.read, like _watch_stderr)
and schedule _handle_gpio_line back onto the loop.
registerPiFamilyKind + ProBoardDef.piFamily route an overlay board through
the existing Raspberry Pi bridge path (WebSocket qemu, VFS panel, boot
terminal); register_pi_board_profile lets the overlay add the matching
PI_CONFIGS entry (cpu/image_set/...) at registration time.
Un crash entre ambos (el NameError de board_fqbn) dejaba defaults==rendered
con un sdkconfig stale, y el re-seed de kconfig no disparaba nunca mas en ese
dir persistente — medido: CONSOLE_SECONDARY quedo en usb_serial_jtag pese a
los defaults nuevos. El bloque de velxio_board.cmake pasa a despues.
El bloque de velxio_board.cmake usaba board_fqbn, que vivia en compile() y no
en la firma de _compile_in_dir: parametro nuevo con default None, propagado en
los dos call sites, y guard para el camino sin fqbn (tests directos).
Tres piezas:
- _arduino_board_flags lee <id>.build.board y <id>.build.cdc_on_boot de
boards.txt (cache, mismo patron que _arduino_variant).
- Cada compile escribe velxio_board.cmake (incluido OPTIONAL por el
CMakeLists ANTES de project(), asi los defines llegan a TODOS los
componentes — que Serial sea el CDC se decide dentro del core Arduino):
ARDUINO_<BOARD> siempre que boards.txt lo nombre (los sketches de vendor
hacen #ifdef ARDUINO_XIAO_ESP32S3), y ARDUINO_USB_CDC_ON_BOOT=1 +
ARDUINO_USB_MODE=1 cuando cdc_on_boot=1. USB_MODE va FORZADO a 1 (HWCDC
por USB-Serial-JTAG, la opcion de menu USBMode=hwcdc): el motor modela esa
consola, no la pila OTG/TinyUSB. Ir por archivo y no por env hace el flag
dependencia de configure: CMake reconfigura al cambiar.
- CONFIG_ESP_CONSOLE_SECONDARY_NONE en el sdkconfig: sin ella el ROM/IDF
espejan cada byte de log al USB y el monitor unico del emulador (que
mezcla ambos streams) mostraria el log entero dos veces en builds CDC.
_normalize_options rellena arduinoVariant con _arduino_variant(board_fqbn),
que ya leia boards.txt; la plantilla ya consumia la clave. Sin esto todas las
placas S3 compilaban como esp32s3 generico: sin nombres Dx del XIAO y con
Serial en UART0, cuyo RX por defecto es GPIO44 = D7 del XIAO — un
pinMode(D7) tardio desinstala el driver y silencia los prints del loop
(medido en el banco con el ejemplo del Round Display).
Sin esta linea la variante se queda en el nombre del chip, asi que TODA placa
compila como dev-kit generico y variants/<VARIANTE>/pins_arduino.h nunca entra
en el include path: los nombres propios de la placa (D0..D10 en una XIAO, la
serigrafia de las M5) sencillamente no existen. Justo los sketches que la gente
copia de la wiki del fabricante.
El helper que deriva la variante del FQBN leyendo boards.txt ya estaba; faltaba
la plantilla que la consume. Sigue pendiente rellenar 'arduinoVariant' en
normalized para que surta efecto (ver F7.6 en velxio-prod).
Los dos ejemplos de camara (esp32cam-lcd-preview y esp32cam-webcam-demo) morian
con "esp_camera.h: No such file or directory".
No es que el ejemplo estuviera mal: es una regresion de la migracion. El arbol
viejo /opt/arduino-esp32 (2.x) traia un SDK precompilado con esp32-camera dentro,
en tools/sdk/<target>/include/esp32-camera/, para esp32, c3, s2 y s3. El
componente 3.x que usamos ahora NO lo trae, y su idf_component.yml tampoco
depende de el, asi que el header simplemente dejo de existir. Por eso funcionaba
antes y dejo de funcionar sin que nadie tocara el ejemplo.
Mezclar los headers de la 2.x en este build no es opcion — vienen con su SDK
precompilado y aqui compilamos contra IDF 5. Lo correcto en el mundo 3.x es
pedirle el componente al gestor: la plantilla no traia idf_component.yml, asi que
todo lo gestionado llegaba de rebote por el manifiesto de arduino-esp32, y lo que
el no arrastra hay que pedirlo explicitamente.
_detect_camera_usage() ve el include y _add_managed_components() escribe
main/idf_component.yml con espressif/esp32-camera. Verificado en el contenedor
sobre el mismo directorio que fallaba: el gestor descarga 2.1.7, el configure
termina con esp32-camera en la lista de componentes, y sketch.ino.cpp compila.
Arduino trata igual #include "Lib.h" que #include <Lib.h> para librerias, y
los ejemplos de los propios fabricantes usan la forma con comillas: los de
M5Stack para el Cardputer abren con #include "M5Cardputer.h". Al escanear solo
la forma con angulos, ese sketch no llegaba siquiera al resolutor de librerias y
moria con 'fatal error: M5Cardputer.h: No such file', mientras el mismo sketch
con angulos compilaba sin problema. Cualquiera que pegue un ejemplo oficial se
daba de bruces con esto.
Se pasan ademas los nombres de los ficheros del propio sketch, para que un
#include "Common.h" local siga siendo una cabecera del proyecto y no se
confunda con una libreria.
test/backend/unit/test_espidf_compiler.py cubre ambas formas, el espaciado, las
cabeceras internas de IDF y la cabecera propia del proyecto.
Las dos ramas habian divergido: master llevaba el modo lenguaje ESP-IDF puro
(#139) y v3.2 la ruta de compilacion IDF v5.5 para toda la familia ESP32 mas
los arreglos de venv/toolchain. Ambas tocaban espidf_compiler.py.
Los dos lados son ejes ORTOGONALES y se conservan enteros:
- use_idf5 / arduino_mode (v3.2): que arbol IDF usa el build (5.5 vs 4.4) y
si cabe Arduino-como-componente.
- pure_idf (master): el modo LENGUAJE que elige el usuario; sus ficheros son
las fuentes del componente main con su propio app_main().
Resolucion:
- _build_env acepta los tres. Un build IDF puro fuerza arduino_mode a falso:
la plantilla CMake mete el componente arduino-esp32 en cuanto existe
ARDUINO_ESP32_PATH, asi que dejarlo puesto compilaba el core de Arduino en
un build que no tiene sketch. Lo cazaron los tests de master.
- VELXIO_PURE_SKETCH solo con pure_idf, nunca con arduino_mode a falso a
secas: un target sin core arduino-esp32 sigue entregando un SKETCH al
traductor legacy y no debe tomar la rama del glob puro.
- La identidad del build-dir suma los dos tokens (|idf:N|ard:N y |lang:pure):
ningun par de esas combinaciones puede compartir un build/ configurado.
- La cadena de escritura de fuentes queda pure_idf -> arduino_mode -> legacy.
- sdkconfig: render de v3.2 (con target/use_idf5) mas el filtrado de simbolos
CONFIG_ARDUINO* de master cuando el build es puro.
test/backend/unit/test_espidf_compiler.py: los 7 tests que ya estaban rotos en
v3.2 (AttributeError: idf5_path, fixture sin actualizar desde que se anadio la
seleccion de IDF) vuelven a pasar.
Verificado: backend 293 pasan / 0 fallan (v3.2 traia 7 rotos); frontend 2268
pasan / 0 fallan en los dos shards.
Adds a third entry to the board language selector next to Arduino C++
and MicroPython: ESP-IDF. In this mode the user writes a plain ESP-IDF
project — app_main() entry point, FreeRTOS + driver APIs — and the
backend compiles it through the same ESP-IDF toolchain it already uses
for ESP32 Arduino sketches, just without the arduino-esp32 component.
Backend:
- CompileRequest.language ('espidf') threaded through the sync + async
compile paths and folded into the dedup job key (language='arduino'
and omitted hash identically so old clients keep dedupping).
- espidf_compiler: pure_idf flag. User files are written into main/
as-is (no Arduino.h wrap, no velxio_compat.h, Arduino library
resolution skipped), ARDUINO_ESP32_PATH is dropped from the build env
and VELXIO_PURE_SKETCH raised so the template CMake compiles the
user's own sources via a glob branch. Pure builds get their own
persistent build-dir variant through the eff_hash fold.
- QEMU WiFi compat for IDF-style code: esp_wifi.h/esp_wifi_init
detection sets has_wifi, and literal #define SSID/PASS plus
wifi_config_t designated initializers are normalized to the QEMU AP.
- CONFIG_ARDUINO_* lines are stripped from sdkconfig.defaults in pure
mode (the symbols don't exist without the arduino component).
Frontend:
- LanguageMode gains 'espidf'; BOARD_SUPPORTS_ESPIDF covers the ESP32
family (Xtensa, S3, C3). Toolbar shows the option only for those.
- Switching modes seeds a main.c blink skeleton (app_main + gpio
driver), mirroring the MicroPython main.py flow.
- compileCode sends language='espidf'; run/stop paths are unchanged
(the QEMU worker consumes the same merged flash image).
- New gallery example: esp32-idf-blink (LED + resistor on GPIO 2).
Tests: unit coverage for the build-env switch, IDF wifi normalization,
job-key variance, file-group seeding and the new example; verified
end-to-end in a container from the prod image (pure build produces a
bootable flash image; Arduino-mode build unchanged, same variant hash).
David Montero Crespo