mcp 2.0.0 shipped on 2026-07-28 and deleted mcp.server.fastmcp, the module
app/mcp/server.py imports. requirements.txt asked for `mcp>=1.0.0` with no
ceiling, so the very next CI run resolved the new major and died importing
test_mcp_tools.py.
That failure is a COLLECTION error, not a test failure: pytest aborts with
exit 2 before running anything, so all 295 unit tests have been silently
unrun on every push since. The workflow was red, but red for a reason
nobody's code caused.
The deploy venv escaped only by accident — it was built while 1.x was still
current, and deploy.sh rebuilds it when requirements.txt changes, so the
next rebuild would have walked into the same wall and taken the deploy gate
down with it.
Verified in a clean venv built from this file: resolves to mcp 1.29.0,
322 passed / 18 skipped, test_mcp_tools.py's 35 tests among them.
Raising the cap means porting to the 2.x server API (mcp.server.mcpserver).
useProjectStore.currentExampleId — stamped by loadExample, dropped the
moment a real project (or a blank workspace) takes over. The compile
request and the run event carry it as example_id, analytics only, so
'which examples get compiled and run most' becomes answerable.
Several boards share one FQBN (the Pimoroni RP2350 trio all compile as
rpipico2), so FQBN-keyed metrics can't tell them apart. The compile
request and the run event now carry board_kind — analytics only, rides
into the metric hook's extra.
suppressOnlineOnlyAds(): overlay seam so the hosted build can silence
'available in the online editor' ad cards for items it manages — an
embargoed item must vanish, not advertise the site the user is on.
Pure OSS builds never call it; ads behave exactly as before.
Declared inside the has_arduino branch, so every successful pure
ESP-IDF (language=espidf) build crashed with UnboundLocalError when the
success return read it. Regression from today's manifest auto-migration
change; the Arduino paths were live-verified but the espidf mode was
not.
NewsMarkdown (shared renderer): YouTube links render as a thumbnail +
play badge that swaps to the youtube-nocookie iframe on click (nothing
loads from the player until the user opts in); images render inline;
all links open in a new tab. Product decision supersedes the alt-text
image stripping: posts embed screenshots/demos, and the README now
documents that embedded media loads in the viewer's browser.
Security review follow-ups (all LOW):
- proxy: negative cache on upstream failure (no outbound fetch per
request while velxio.dev is unreachable) and a 2MB body cap via
streaming read (httpx timeouts are per read op, not per body)
- NewsModal: OSS builds render markdown images as alt text so viewers'
browsers never contact remote hosts; pro cloud builds show them
Editor shows scheduled product-news posts as a markdown modal: max one
per day, oldest first, each post only ever once (marked seen at
delivery). OSS default source proxies velxio.dev's public feed through
the local backend (cached ~6h, anonymous, VELXIO_NEWS=off to disable)
with seen-state in localStorage; a pro overlay can register a
server-backed per-user source via lib/newsSource.ts.
File names may now carry '/' paths; the explorer renders each board group
as a collapsible folder tree (folders first, files in group order, CSS-var
indentation because .fe-file-item pins padding with !important). New-folder
buttons on the board header and per-folder hover actions (new file here,
new subfolder, delete with confirm when files are inside; a folder holding
every file of the group cannot be deleted - groups keep >= 1 file).
Model: names are the source of truth; only EMPTY folders are tracked
(folderGroups per group) so they survive until a file lands in them, then
materialise into the name prefix. Round-trips through .vlx (optional
folderGroups field, laxly validated), the auth stash draft, and
loadProjectState. Examples gain boards[].files so a single board can ship
a full multi-file tree (first entry = main sketch).
Backend: the arduino-cli sketch writer now creates subdirectories for
'/'-pathed names and REJECTS names that resolve outside the sketch dir
('..' traversal) - previously it crashed on the former and wrote outside
the tmp dir on the latter.
Three-part fix for the library-contamination class of ESP32 compile
failures (a user's project kept pulling unrelated libraries because 79%
of projects compile with an empty manifest, forcing scan-all resolution
over ~1000 shared libraries):
1. Name-affinity ranking in _find_library_for_header: the provider whose
name matches the header stem wins (exact normalized match > esp32
arch declaration > contains-stem, alphabetical only as tie-break).
Previously first-alphabetical won, so any stray library shipping a
generic header (Servo.h has 6 providers) could hijack the build.
2. Manifest auto-migration: scan-all compiles now report the libraries
the build really used (manifest_incomplete +
manifest_suggested_libraries on the compile result); the editor folds
single-candidate suggestions into the board's declared manifest after
a green build, so the NEXT compile runs scoped. One green build = one
migrated project; empty-manifest scan-all behaviour is untouched.
3. Install-declares: the Wokwi import library modal records successful
installs in the active board's manifest, and .vlx export/import now
round-trips board.libraries.
Tests: test_espidf_library_ranking.py (9) + library-manifest.test.ts (19).
Reported 2026-08-03: every ESP32 build in a user's project failed with
errors from libraries he never installed, emitted from user_libs_all --
a folder his file list does not even show, in projects he had recreated
from scratch.
Root cause: _detect_external_includes was a bare regex with no
preprocessor state. ESPAsyncWebServer guards <Hash.h> behind
'#if defined(ESP8266) || defined(TARGET_RP2040)', so the compiler never
sees it on ESP32 -- but the scanner reported it anyway. The resolver
then searched the ~1000 shared-cache libraries, matched stemi-hexapod's
src/Hash.h (a hexapod-robot library), merged it WHOLE, and compiled all
of its sources; each failed on its own unrelated includes.
The scanner now tracks #if/#ifdef/#ifndef/#elif/#else/#endif nesting and
strips comments. A branch is skipped ONLY when every identifier in its
condition is a platform macro known to be undefined on ESP32; unknown
macros keep the branch LIVE, because hiding a genuinely needed header
would be a much worse failure than the old over-reporting. Verified
against the real ESPAsyncWebServer shape plus nesting/elif/else/comment
cases (9 new tests, 15 green in the espidf suites).
Complements the platform-header guard from the previous commit, which
stays as defence in depth.
Reported by a user whose every ESP32 build failed with errors from
libraries he never installed (SoftwareSerial.h, ArduinoBearSSL.h,
Adafruit_GFX.h, AsyncElegantOTA.h) coming out of user_libs_all, in a
project whose file list doesn't even show that folder.
Traced end to end: ESPAsyncWebServer includes <Hash.h> (an ESP8266-core
header, guarded for that platform, so arduino-esp32's tree has no such
file). The global scan then resolved it to stemi-hexapod's src/Hash.h --
a hexapod-robot library -- merged the WHOLE library into user_libs_all,
and its Serial.cpp / Hash.cpp / Server.cpp / BluetoothLowEnergy.cpp /
ExpansionDriver.cpp were compiled, each failing on its own unrelated
includes. Log line: 'Merging "stemihexapod" into user_libs_all for
<Hash.h>'.
Hash.h and friends now join the never-resolve-against-user-libs set.
They cannot be discovered by the core-tree scan (the core genuinely
lacks them) yet their names are generic enough that some installed
library owns a file by that name.
Installing ArduinoJson made the project stop compiling, with an error that
never mentioned ArduinoJson:
CMake Error at user_libs/user_libs_all/CMakeLists.txt:15
(target_compile_definitions):
target_compile_definitions may only set INTERFACE properties on
INTERFACE targets
-- Configuring incomplete, errors occurred!
ArduinoJson ships headers only. With no sources, idf_component_register()
builds the merged user-libs component as an INTERFACE library, and PRIVATE
scope on an INTERFACE target is a hard CMake error - so the configure step
died before a single file was compiled. The user saw only a failing build
and reported it as "ArduinoJson can't be found even after it has been
installed" (#124).
Those definitions map alloca/memcpy_P/memcmp_P for library C sources that
use them undeclared; with no sources there is nothing to define, so they are
emitted only when the component actually compiles something. A library that
carries its own .cpp is unchanged - which is why LiquidCrystal_I2C (#257)
worked and hid this.
Two-tier rule replacing the blanket skip:
- 'all' now counts as wildcard ('*'): LiquidCrystal_I2C 2.0.0 declares
architectures=all and was skipped on every ESP32 build, breaking the
most popular I2C LCD library with a bare 'No such file or directory'.
- Headers the SKETCH includes directly merge even when the library
declares a foreign-only architecture (many avr-declared libs are pure
Wire/SPI code that compiles fine; a truthful compile error beats a
missing-header one). Transitive pulls keep the HARD guard, which is
the path that dragged SAMD-only Adafruit_ZeroDMA into ESP32 builds.
Lets metrics overlays distinguish agent-triggered compiles from manual
ones (the AI assistant's compiles were previously invisible in project
counters). Loose optional field — omitted means manual.
Cada guest es un proceso QEMU con su propia RAM (1-2 GB segun placa) y
sus hilos de vCPU, asi que el limite lo pone la maquina, no el codigo.
No habia ningun tope: el usuario N simplemente empujaba la caja a swap y
la sesion de TODOS se volvia lenta, que es peor que decirle al usuario N
que espere un minuto.
VELXIO_PI_MAX_INSTANCES (6) guests simultaneos en la maquina
VELXIO_PI_MAX_PER_OWNER (2) guests por persona
El "owner" es el hash de la cookie de sesion: sirve solo para contar, no
se guarda ni se lee de vuelta, y cae al host del cliente cuando no hay
cookie (sidecar de escritorio, tests). Sin identidad solo aplica el tope
global.
El rechazo llega como un mensaje que dice que hacer ("prueba en un
minuto" / "para una de tus sesiones"), no como un fallo mudo, y se
comprueba ANTES de lanzar el proceso.
Auditoria de "la Pi tiene todo lo de la placa real" con cuatro huecos
encontrados y cerrados:
1) GPIO de entrada en modo Linux: GPIO_IN respondia VAL 0 fijo (stub de
la fase 2), asi que GPIO.input() leia 0 eternamente aunque el canvas
empujara el nivel. El backend guarda ahora el ultimo nivel por pin
(set_pin_state lo escribe) y GPIO_IN contesta de ahi. Los flancos
(SET) siguen llegando al guest como antes.
2) UART del header hacia otra placa: el shim del rootfs ya hablaba
`UART <port> TX <hex>` / RX_REQ, pero sin modelo de esclavo el
backend tragaba los bytes. Ahora TX sin esclavo se emite al canvas
(uart_tx) y RX_REQ sin esclavo drena la cola que llena pi_uart_rx —
el mismo protocolo de siempre, sin ops nuevas.
3) El escaner de esclavos I2C/SPI/UART estaba doblemente muerto:
clasificaba por numero fisico de pin ('3','5','19'...) cuando el
elemento expone GPIOxx, y su unico llamador era RaspberryPiWorkspace,
que el terminal unificado reemplazo. Acepta ambos nombres y corre en
onBooted del store.
4) boardPinToNumber solo mapeaba los pines de la 3/4/5; la Zero, 1B+ y
2B (mismo header de 40 pines, mismo elemento) se quedaban sin mapa.
La placa QEMU-Linux tiene DOS flujos serie y hasta ahora el cableado
usaba el equivocado: el enrutado entregaba los bytes del vecino a la
consola (el shell) y sacaba al cable la cháchara del arranque. El
header, que es lo que el usuario cablea, no existia.
Ahora el canal de protocolo lleva dos ops nuevas:
UARTTX <b64> el guest transmitio por el header -> al canvas
UARTRX el guest pregunta que le llego -> UART_RXQ <b64>
El backend guarda una cola por instancia (acotada a 64 KB, que un script
que no lee nunca no la haga crecer) y el websocket acepta `pi_uart_rx`
con los bytes que el vecino manda. En el frontend el bridge gana
onUartTx / sendUartBytes y el Interconnect engancha ESE flujo en vez de
la consola para las placas Pi.
Con esto el mismo script -- import serial, escribir, dormir, leer --
funciona en los dos motores.
Guests stay on '-nic none' unless a profile opts in, and opting in does
NOT mean internet: the NIC is user,restrict=on (no route out, no route
to the host LAN) with one guestfwd to whatever command the overlay
configures — a filtering proxy in practice. Keeps 'user code never gets
a raw socket outside' true by construction.
A forgotten tab pinned a QEMU process and its RAM for as long as the
browser stayed open. Instances now shut themselves down after
MAX_SESSION_SECONDS (2 h default, env-tunable) and say so on the serial
line instead of vanishing.
A profile's extra_drive is the same file for everyone; an overlay may
need a disk built for THIS session (what the project declared). New
seam set_pi_extra_drive_resolver(fn) receives the client id, the board
and the start_pi payload and returns raw images, mounted read-only after
the profile's own. The WS route forwards msg_data and the bridge gained
startPayload so a client can declare it. Generic: no package manager,
no OS knowledge in the OSS tree.
- profile key extra_drive: optional read-only second virtio-blk so an
overlay can ship guest-side shim libraries (/dev/vdb)
- SENS <name> protocol op: canvas-fed named values (built-in sensors /
buttons) served from PiInstance.sensor_state, pushed by the frontend
via the new pi_sensor_state WS message
- DISP <b64> protocol op: guest display commands forwarded to the
frontend as 'display' events (built-in screens)
- RaspberryPi3Bridge: onDisplay / onGpioPwm callbacks + setSensorState
- SimulatorCanvas hands piFamily boards their Pi bridge in
attachBuiltins (was ESP32-only)
add_reader on the proto FIFO armed epoll but never delivered callbacks
under uvloop when the fd number recycled a just-closed socket fd
(nondeterministic per instance): the guest's GPIO lines sat unread in
the pipe and canvas LEDs stayed dark while serial kept flowing. Pump
the FIFO from a worker thread (select + os.read, like _watch_stderr)
and schedule _handle_gpio_line back onto the loop.
registerPiFamilyKind + ProBoardDef.piFamily route an overlay board through
the existing Raspberry Pi bridge path (WebSocket qemu, VFS panel, boot
terminal); register_pi_board_profile lets the overlay add the matching
PI_CONFIGS entry (cpu/image_set/...) at registration time.
Un crash entre ambos (el NameError de board_fqbn) dejaba defaults==rendered
con un sdkconfig stale, y el re-seed de kconfig no disparaba nunca mas en ese
dir persistente — medido: CONSOLE_SECONDARY quedo en usb_serial_jtag pese a
los defaults nuevos. El bloque de velxio_board.cmake pasa a despues.
El bloque de velxio_board.cmake usaba board_fqbn, que vivia en compile() y no
en la firma de _compile_in_dir: parametro nuevo con default None, propagado en
los dos call sites, y guard para el camino sin fqbn (tests directos).
Tres piezas:
- _arduino_board_flags lee <id>.build.board y <id>.build.cdc_on_boot de
boards.txt (cache, mismo patron que _arduino_variant).
- Cada compile escribe velxio_board.cmake (incluido OPTIONAL por el
CMakeLists ANTES de project(), asi los defines llegan a TODOS los
componentes — que Serial sea el CDC se decide dentro del core Arduino):
ARDUINO_<BOARD> siempre que boards.txt lo nombre (los sketches de vendor
hacen #ifdef ARDUINO_XIAO_ESP32S3), y ARDUINO_USB_CDC_ON_BOOT=1 +
ARDUINO_USB_MODE=1 cuando cdc_on_boot=1. USB_MODE va FORZADO a 1 (HWCDC
por USB-Serial-JTAG, la opcion de menu USBMode=hwcdc): el motor modela esa
consola, no la pila OTG/TinyUSB. Ir por archivo y no por env hace el flag
dependencia de configure: CMake reconfigura al cambiar.
- CONFIG_ESP_CONSOLE_SECONDARY_NONE en el sdkconfig: sin ella el ROM/IDF
espejan cada byte de log al USB y el monitor unico del emulador (que
mezcla ambos streams) mostraria el log entero dos veces en builds CDC.
_normalize_options rellena arduinoVariant con _arduino_variant(board_fqbn),
que ya leia boards.txt; la plantilla ya consumia la clave. Sin esto todas las
placas S3 compilaban como esp32s3 generico: sin nombres Dx del XIAO y con
Serial en UART0, cuyo RX por defecto es GPIO44 = D7 del XIAO — un
pinMode(D7) tardio desinstala el driver y silencia los prints del loop
(medido en el banco con el ejemplo del Round Display).
Sin esta linea la variante se queda en el nombre del chip, asi que TODA placa
compila como dev-kit generico y variants/<VARIANTE>/pins_arduino.h nunca entra
en el include path: los nombres propios de la placa (D0..D10 en una XIAO, la
serigrafia de las M5) sencillamente no existen. Justo los sketches que la gente
copia de la wiki del fabricante.
El helper que deriva la variante del FQBN leyendo boards.txt ya estaba; faltaba
la plantilla que la consume. Sigue pendiente rellenar 'arduinoVariant' en
normalized para que surta efecto (ver F7.6 en velxio-prod).
Los dos ejemplos de camara (esp32cam-lcd-preview y esp32cam-webcam-demo) morian
con "esp_camera.h: No such file or directory".
No es que el ejemplo estuviera mal: es una regresion de la migracion. El arbol
viejo /opt/arduino-esp32 (2.x) traia un SDK precompilado con esp32-camera dentro,
en tools/sdk/<target>/include/esp32-camera/, para esp32, c3, s2 y s3. El
componente 3.x que usamos ahora NO lo trae, y su idf_component.yml tampoco
depende de el, asi que el header simplemente dejo de existir. Por eso funcionaba
antes y dejo de funcionar sin que nadie tocara el ejemplo.
Mezclar los headers de la 2.x en este build no es opcion — vienen con su SDK
precompilado y aqui compilamos contra IDF 5. Lo correcto en el mundo 3.x es
pedirle el componente al gestor: la plantilla no traia idf_component.yml, asi que
todo lo gestionado llegaba de rebote por el manifiesto de arduino-esp32, y lo que
el no arrastra hay que pedirlo explicitamente.
_detect_camera_usage() ve el include y _add_managed_components() escribe
main/idf_component.yml con espressif/esp32-camera. Verificado en el contenedor
sobre el mismo directorio que fallaba: el gestor descarga 2.1.7, el configure
termina con esp32-camera en la lista de componentes, y sketch.ino.cpp compila.
Arduino trata igual #include "Lib.h" que #include <Lib.h> para librerias, y
los ejemplos de los propios fabricantes usan la forma con comillas: los de
M5Stack para el Cardputer abren con #include "M5Cardputer.h". Al escanear solo
la forma con angulos, ese sketch no llegaba siquiera al resolutor de librerias y
moria con 'fatal error: M5Cardputer.h: No such file', mientras el mismo sketch
con angulos compilaba sin problema. Cualquiera que pegue un ejemplo oficial se
daba de bruces con esto.
Se pasan ademas los nombres de los ficheros del propio sketch, para que un
#include "Common.h" local siga siendo una cabecera del proyecto y no se
confunda con una libreria.
test/backend/unit/test_espidf_compiler.py cubre ambas formas, el espaciado, las
cabeceras internas de IDF y la cabecera propia del proyecto.
Las dos ramas habian divergido: master llevaba el modo lenguaje ESP-IDF puro
(#139) y v3.2 la ruta de compilacion IDF v5.5 para toda la familia ESP32 mas
los arreglos de venv/toolchain. Ambas tocaban espidf_compiler.py.
Los dos lados son ejes ORTOGONALES y se conservan enteros:
- use_idf5 / arduino_mode (v3.2): que arbol IDF usa el build (5.5 vs 4.4) y
si cabe Arduino-como-componente.
- pure_idf (master): el modo LENGUAJE que elige el usuario; sus ficheros son
las fuentes del componente main con su propio app_main().
Resolucion:
- _build_env acepta los tres. Un build IDF puro fuerza arduino_mode a falso:
la plantilla CMake mete el componente arduino-esp32 en cuanto existe
ARDUINO_ESP32_PATH, asi que dejarlo puesto compilaba el core de Arduino en
un build que no tiene sketch. Lo cazaron los tests de master.
- VELXIO_PURE_SKETCH solo con pure_idf, nunca con arduino_mode a falso a
secas: un target sin core arduino-esp32 sigue entregando un SKETCH al
traductor legacy y no debe tomar la rama del glob puro.
- La identidad del build-dir suma los dos tokens (|idf:N|ard:N y |lang:pure):
ningun par de esas combinaciones puede compartir un build/ configurado.
- La cadena de escritura de fuentes queda pure_idf -> arduino_mode -> legacy.
- sdkconfig: render de v3.2 (con target/use_idf5) mas el filtrado de simbolos
CONFIG_ARDUINO* de master cuando el build es puro.
test/backend/unit/test_espidf_compiler.py: los 7 tests que ya estaban rotos en
v3.2 (AttributeError: idf5_path, fixture sin actualizar desde que se anadio la
seleccion de IDF) vuelven a pasar.
Verificado: backend 293 pasan / 0 fallan (v3.2 traia 7 rotos); frontend 2268
pasan / 0 fallan en los dos shards.
Adds a third entry to the board language selector next to Arduino C++
and MicroPython: ESP-IDF. In this mode the user writes a plain ESP-IDF
project — app_main() entry point, FreeRTOS + driver APIs — and the
backend compiles it through the same ESP-IDF toolchain it already uses
for ESP32 Arduino sketches, just without the arduino-esp32 component.
Backend:
- CompileRequest.language ('espidf') threaded through the sync + async
compile paths and folded into the dedup job key (language='arduino'
and omitted hash identically so old clients keep dedupping).
- espidf_compiler: pure_idf flag. User files are written into main/
as-is (no Arduino.h wrap, no velxio_compat.h, Arduino library
resolution skipped), ARDUINO_ESP32_PATH is dropped from the build env
and VELXIO_PURE_SKETCH raised so the template CMake compiles the
user's own sources via a glob branch. Pure builds get their own
persistent build-dir variant through the eff_hash fold.
- QEMU WiFi compat for IDF-style code: esp_wifi.h/esp_wifi_init
detection sets has_wifi, and literal #define SSID/PASS plus
wifi_config_t designated initializers are normalized to the QEMU AP.
- CONFIG_ARDUINO_* lines are stripped from sdkconfig.defaults in pure
mode (the symbols don't exist without the arduino component).
Frontend:
- LanguageMode gains 'espidf'; BOARD_SUPPORTS_ESPIDF covers the ESP32
family (Xtensa, S3, C3). Toolbar shows the option only for those.
- Switching modes seeds a main.c blink skeleton (app_main + gpio
driver), mirroring the MicroPython main.py flow.
- compileCode sends language='espidf'; run/stop paths are unchanged
(the QEMU worker consumes the same merged flash image).
- New gallery example: esp32-idf-blink (LED + resistor on GPIO 2).
Tests: unit coverage for the build-env switch, IDF wifi normalization,
job-key variance, file-group seeding and the new example; verified
end-to-end in a container from the prod image (pure build produces a
bootable flash image; Arduino-mode build unchanged, same variant hash).
Ports the v5 compiler layer: /opt/esp-idf-v5 + arduino-esp32 3.x
discovery (IDF5_PATH / ARDUINO_ESP32_5_PATH overridable), per-compile IDF
selection routing the family to the v5 tree, esp32c6 target support, and
the docker-runtime hardening (version-matched python venv pinned on PATH,
toolchain ordering by the tree's tools.json, user-libs component REQUIRES
for the IDF components Arduino libraries include directly). The LED_BUILTIN
fallback in the project template becomes a whitelist so non-esp32 variants
(C6 first) never hit the redefinition trap. Includes the option tests.
The newest-env-wins heuristic pinned a 5.x env for a 4.4 tree — each
generation's dependency check only accepts its own requirement set. The
version is read from tools/cmake/version.cmake of the active IDF tree,
falling back to the newest env when unreadable.
Two failure modes seen on a fresh docker deploy where the final image's
system python differs from the one the IDF venv was created with:
- IDF's cmake derives the venv dir name from the SYSTEM python version
and fails with 'python doesn't exist' before configure; and when cmake
is invoked directly (not via idf.py) its PYTHON property falls back to
the bare 'python' from PATH. The Linux branch of _build_env now pins
IDF_PYTHON_ENV_PATH to the newest env under the tools root and puts its
bin first on PATH (mirroring what the Windows branch already did).
- When several toolchain generations share one tools root, glob order
decided which won the PATH race. Toolchain dirs are now ordered by the
versions the IDF tree declares in tools/tools.json.
drazzy.com (ATTinyCore index host) has had an expired TLS certificate
since 2026-06-22 and now 301-redirects http to https, defeating the
plain-http URL pinned to sidestep its TLS issues. Two failure modes:
1. A /root/.arduino15 volume from an older image can reference the index
in config while lacking the file; arduino-cli then fails instance
init outright, breaking EVERY compile, not just ATtiny (issue #254).
2. backend/Dockerfile chained update-index with &&, so any uncached
image build fails hard while the host is broken (exit 1 verified).
A stale index is harmless (ATTinyCore 1.4.1's platform archive and its
micronucleus 2.0a4 both download from github.com); a missing one is
fatal. So: vendor the index under backend/board-indexes/, copy it to
/opt/arduino15-seed in Dockerfile.standalone, and teach entrypoint.sh
to seed any missing package_*.json into /root/.arduino15 at boot,
healing stale volumes. backend/Dockerfile seeds the index directly and
makes update-index best-effort; core install lines stay strict.
Verified: removing the index reproduces the reporter's exact
'Error initializing instance' brick; after seeding, instance init
exits 0 with the host still broken.
_emit() wrote to stdout synchronously from QEMU callback context — the
iothread fires _on_uart_tx per UART byte, each becoming ~45 bytes of
JSON. When the parent's pipe reader stalled, the 64 KB pipe filled and
the write blocked INSIDE the QEMU iothread, freezing the entire guest.
Observed as an intermittent (~1 in 10) boot hang: serial output stops
right after the ROM log / rtcinit line — exactly where the accumulated
boot events cross the pipe capacity — and never recovers. Direct worker
harness runs (fast reader) never reproduced it; the browser/WS path did.
Route _emit() through a bounded queue drained by a dedicated writer
thread (opportunistic batching, one write per drain). Under extreme
backpressure events are dropped and counted on stderr — losing telemetry
beats freezing the emulated CPU. The shutdown path posts a sentinel and
joins the writer so crash/system events still flush before os._exit.
Verified on staging: 10/10 UI run/stop cycles + 8/8 direct harness boots
with identical serial latency (~0.95s to first app output).
_refresh_pin_pulls read a fixed 40 registers from get_internals(3), but
the exposed array is per-chip: classic muxgpios[40], ESP32-S3 49
(GPIO0..48). The fixed bound silently missed S3 pulls on GPIO40-48.
Read _GPIO_COUNT entries instead (set from the machine at startup).
Pairs with qemu-lcgamboa 547e989, which models the S3 IO_MUX pull bits
and exposes them via the get_internals override — together they make
INPUT_PULLUP buttons work on the emulated S3 exactly like the classic
ESP32 (verified on staging with GPIO4 and GPIO40).
The S3 has GPIO0..48; the identity pinmap length drives how many pins
picsimlab_wire_gpio connects to the host. Default 40 left GPIO40..48 unwired,
so digitalWrite/Read on those pins never reached the frontend. Build a 49-entry
pinmap when the machine is esp32s3 (mirrors the c3->22 special case). Requires
the GPIO-model widening in libqemu (esp32_gpio bank-1 8->17).
_idf_target had no S3 case, so every S3 FQBN (esp32s3 / XIAO_ESP32S3 /
nano_nora) compiled as plain esp32 (LX6) and could not boot the S3 QEMU
machine. Add _is_esp32s3 (covers all three FQBNs; nano_nora has no 's3'
token) + return 'esp32s3'; add the xtensa-esp32s3-elf toolchain to the
build PATH; place the S3 second-stage bootloader at flash offset 0x0
(like C3) instead of 0x1000 via 'is_c3 or idf_target==esp32s3' at the merge.
Requires the xtensa-esp32s3-elf toolchain in the image
(install.sh esp32,esp32c3,esp32s3) and a libqemu-xtensa with an
esp32s3(-picsimlab) machine.
Extend the spice-driven input path (already live for AVR/ESP32) to RP2040 and
STM32 so digitalRead() of an INPUT pin reflects the actual wiring: a pin tied
to a rail reads that rail, and an INPUT_PULLUP button-to-GND reads idle-HIGH /
pressed-LOW instead of floating or inverted.
RP2040 (rp2040js, frontend-only): the GPIO listener now splits input vs output
mode. Input pins report their pad pull (InputPullUp/Down) via setPinPull and
seed the pull's idle level (rp2040js does not auto-apply the pad pull to the
readable input register); the SPICE solve then overrides via connectDigital-
InputsToMcu when the net is actually sourced. Output pins drive as before.
spiceDrivenInputs = true.
STM32 (backend QEMU): the worker now forwards a new gpio_pull event (from the
libqemu-arm picsimlab_pull_pin callback) so the netlist stamps the matching
weak resistor; Stm32Bridge surfaces it, Stm32BridgeShim opts into
spiceDrivenInputs, and collectPinStates maps PA0/PC13 names to the linear pin
so the pull is read. STM32 outputs stay on the part layer (unchanged).
Event-driven parts with no SPICE model (rotary encoder, keypad) remain
protected by the existing sourcedNets gate in the connector.
Completes the internal-pull emulation for the common case (a button on an
RTC-capable GPIO like 4/15/25/... with INPUT_PULLUP):
- Backend reads the RTC_IO pad RUE/RDE bits via the new
get_internals(QEMU_INTERNAL_RTCIO) and emits gpio_pull for RTC pins, so
pull-up/down on those pads is finally visible (it lives in RTC_IO, not
IO_MUX). IO_MUX path still covers non-RTC pins.
- The digitalRead path is driven by seeding the GPIO input level, not by
SPICE. The part-level INPUT_PULLUP seed (BasicParts) is sent at attach,
before the multi-second QEMU boot finishes, so it is lost and the pin
reads LOW. makePinPullHandler now drives the pin to the pull's idle level
via sendPinEvent when the guest programs the pull (post-boot), so it
sticks. A real button press/release still overrides it.
INPUT_PULLUP / INPUT_PULLDOWN had no effect in simulation: the ESP32's
internal pull resistors live inside QEMU and were invisible to the SPICE
solver, so an input wired to a button-to-GND floated to 0 V and read LOW
even at idle. The canonical active-low button never worked.
Read the pull config straight out of the running guest: the IO_MUX
register (FUN_PU bit 8 / FUN_PD bit 7) is already exposed read-only via
qemu_picsimlab_get_internals(3), so no QEMU rebuild is needed. The worker
scans it on the 100 ms poll thread and emits gpio_pull; the bridge feeds
it to PinManager; the netlist stamps a weak 45k resistor to the rail so
idle inputs read the correct level. 45k matches the real internal pull
and is weak enough that any external driver/pull dominates.
Verified with ngspice: idle ~3.3 V (HIGH), pressed ~0 V (LOW).
- boot_images manifest: bump arm64 rootfs (gpiozero/colorzero baked in,
hostname applied at boot, reworded MOTD)
- RaspberryPi3Bridge: onBooted shell-ready detector + sendAndWaitForPrompt
flow control (resets on disconnect)
- RaspberryPiWorkspace: distinct Booting overlay + piBooted-driven status,
inline SVG icons replacing emoji glyphs
- SerialMonitor: strip CSI/DSR escapes so the dumb console no longer shows
a literal [6n next to the prompt
- VirtualFileSystem: upload auto-starts the Pi and waits for the shell, then
flow-controls each command (no more dropped lines on large files)
- i18n: bootingTitle/bootingNote + reworded offlineNote2 across 9 locales
Move the CYW43439 (Pico W) WiFi emulation out of the open-source tree so it
can ship as a paid feature in a private overlay. OSS keeps a plain Pico W
(no WiFi); the overlay registers the cyw43 protocol + backend network stack
at runtime via generic seams.
Frontend:
- Add simulation/PioPeripheral.ts: a generic "PIO bus peripheral" seam
(feedWord / inDiscardableWriteData / resetFraming / hostWakeLevel /
onHostWake / onSimulationStart). No factory is installed in OSS, so
createPioPeripheral() returns null and a Pico W simulates as a plain Pico.
- RP2040Simulator: keep the fragile PIO-FIFO plumbing (it must re-run after
loadMicroPython swaps the chip) but drive it through PioPeripheral instead
of an inlined cyw43 import (attachCyw43 -> attachPioPeripheral, etc.).
- useSimulatorStore: generic attach/detach + setBoardWifiStatus; drop the
cyw43 bridge map.
- MicroPythonLoader: add registerFirmwareVariant() so an overlay can add the
RPI_PICO_W build; remove the OSS pico-w config + bundled .uf2.
- Delete simulation/cyw43/ (moved to the overlay).
Backend:
- core/hooks.py: add generic register_ws_sim_handler / dispatch_ws_sim_message
and register_gateway_proxy / dispatch_gateway_proxy seams.
- simulation.py: route start_picow / stop_picow / picow_packet_out through the
ws_sim_handler hook (the overlay handles + gates them).
- iot_gateway.py: resolve the Pico W gateway through the gateway_proxy hook.
- Delete services/picow_net/ + picow_net_bridge.py (moved to the overlay).
Tests: move the cyw43/picow suites to the overlay; update RP2040Simulator
mock stubs to attachPioPeripheral.
Two real-world failures hit the Pico W gateway that the headless e2e
(node fetch, tiny request, fast timing) didn't surface:
- A browser sends KILOBYTES of headers (cookies, User-Agent, sec-*).
Forwarded verbatim, the request overran the chip's small recv()
(e.g. recv(1024)); lwIP then RST the connection on close-with-unread-
data, crashing blocking-socket sketches with ECONNRESET. Now we forward
a MINIMAL request (method, path, Host, Connection: close, and
Content-Type/Length for bodies) — nothing a tiny server can choke on.
- After a gateway request completed we dropped the connection immediately,
so a late chip segment (retransmitted FIN / trailing ACK) no longer
matched and fell through to the chip-initiated NAT, which RST it. Add a
short TIME_WAIT: keep the connection briefly and re-ACK late segments so
the chip closes cleanly, never RSTing it.
ESP32 web-server examples are reachable from the browser via
/api/gateway/<client_id>/ (QEMU slirp hostfwd). The Pico W server lives
in the browser-side lwIP, so there was no inbound path: visiting the
chip's IP did nothing.
Add the mirror of tcp_nat.py: tcp_inbound.TcpInbound originates a TCP
connection INTO the chip over the WebSocket bridge (SYN -> SYN+ACK ->
ACK -> request -> response -> FIN), so the backend can fetch a page the
sketch serves on 10.13.37.42:80 and hand it back to the browser.
- bridge.py routes chip TCP segments addressed to a gateway-opened
connection to TcpInbound (before the chip-initiated NAT, which would
RST them); exposes http_into_chip() + ensure_chip_mac() (primes the
chip's gateway ARP).
- iot_gateway.py: same /api/gateway/<client_id>/ route now falls through
to the Pico W bridge when there's no ESP32 instance, builds a raw
HTTP/1.1 request, and parses the chip's response. Same plan gate, same
URL shape — the browser sees no difference between ESP32 and Pico W.
Validated end to end (real RP2040 emulator serving an HTTP page ->
gateway returns it) plus 6 unit tests for the TCP state machine,
response parsing and ARP priming.
Add a working microSD card part backed by a FAT16 image, following the
Wokwi storage model: the project's own workspace files are auto-copied
onto the card (free), and an optional "SD Card" panel uploads extra
files (gated as a paid feature by the velxio.dev overlay; OSS default
allows it).
Frontend (in-browser AVR / RP2040):
- ProtocolParts.ts: rewrite the microsd-card part from a handshake stub
into a real SD-over-SPI device (reply-first Ncr timing, SDSC byte
addressing, single/multi-block read+write, CSD/CID, full CMD set).
- utils/fatImage.ts: dependency-free FAT16 super-floppy builder (8.3 + LFN).
- utils/sdCardFiles.ts: assemble the card image from workspace files plus
uploaded files; base64 helpers.
- components/simulator/SdCardPanel.tsx + ComponentPropertyDialog: upload UI.
- DynamicComponent + useSimulatorStore: build and inject the image on run.
- lib/proSdCardGate.ts: overlay-installable gate for the upload action.
- data/examples-storage-microsd.ts: Arduino Uno + ESP32 gallery examples.
Backend (ESP32 via QEMU):
- services/esp32_sd_slave.py: synchronous SD-over-SPI slave (Python port of
the browser part) with a sparse backing store, idle-state R1 tracking and
real CRC16 on data blocks when the host enables CRC (CMD59) -- both
required by ESP-IDF's sdspi driver.
- esp32_worker.py: route SPI bytes to the slave (returns MISO synchronously)
and feed write-only bulk transfers.
- esp32_lib_manager.py + routes/simulation.py: forward the FAT image
(sd_card.image_b64) from the start config into the worker.
Tested:
- frontend: protocol-parts, fat-image, sd-card-gate and microsd-real-firmware
(real Arduino SD.h on avr8js) -- 86 passing.
- backend: test_esp32_sd_slave (10) covering the ESP-IDF init sequence and
CRC16; validated end to end by running a real SD.h sketch in libqemu-xtensa
(mount, directory listing, read and write-readback).
David Montero Crespo