velxio/test/test-esp32-cam/autosearch/06_existing_test_patterns.md

119 lines
5.4 KiB
Markdown
Raw Normal View History

feat: ESP32-CAM emulation with webcam frame bridge First open-source end-to-end emulation of the AI-Thinker ESP32-CAM in QEMU, paired with a browser webcam → firmware bridge so users can develop camera sketches without hardware. Status: esp_camera_init() returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by the upstream driver. Final piece (cam_task accepting frames) is in progress — descriptor walker fix landed in this commit. Backend (Python/FastAPI): - simulation.py: camera_attach/frame/detach WS handlers - esp32_worker.py: ctypes binding to velxio_push_camera_frame + feature-detection fallback for older DLLs - esp32_lib_manager.py: forward camera commands to the worker stdin - esp-idf-template/main/CMakeLists.txt: esp32-camera headers added via add_prebuilt_library + REQUIRES driver (resolves i2c_master_* symbols). LED_BUILTIN=2 fallback for sketches that hardcode it. Frontend (React/TS): - EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family) added to isQemuBoard list — Run button now starts the QEMU bridge for these boards instead of falling through to the AVR path - useWebcamFrames.ts: getUserMedia → OffscreenCanvas → toBlob('image/jpeg') → base64 → WS at ~10 fps - CameraToggle.tsx: header button with status colors + frame counter - SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards - Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa - useSimulatorStore.ts: diagnostic log on compileBoardProgram - components-metadata.json: regen including esp32-cam component Submodule pointer: - wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32) Investigation + tests in test/test-esp32-cam/: - 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S spec, build blueprint, blockers resolved, descriptor walker fix) - 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip, webcam_demo) + 8 live + WS regression tests - README with the user-facing flow .gitignore: - libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated) - wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path, not part of this repo) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:28:55 +07:00
# 06 — existing ESP32 test patterns we copy
The DHT22 and HC-SR04 simulations have been proving the same
WebSocket+QEMU pipeline that the camera will reuse. This note
extracts the **exact** patterns so the camera tests don't reinvent
anything.
## File layout
| Layer purpose | Where to put it |
|-------------------------------------------|--------------------------------------------|
| Pure unit (Python, mocks the route) | `test/test-esp32-cam/tests/*.py` |
| Frontend integration (Vitest, mocks WS) | `frontend/src/__tests__/esp32-*.test.ts` |
| **Live e2e (Node, real backend, real QEMU)** | `test/backend/e2e/test_*.mjs` |
The live e2e tests in `test/backend/e2e/` use plain Node with
`fetch()` + `WebSocket` (Node 20+). Both the DHT22 and HC-SR04 scripts
are ~250 lines and follow the same skeleton:
1. POST `/api/compile/` with the `.ino` source → get `firmware_b64`.
2. Open `ws://localhost:8001/api/simulation/ws/<session_id>`.
3. On `open`, send `start_esp32` with `{board, firmware_b64, sensors[]}`.
4. Stream `serial_output` messages, buffer until `\n`, scan for
expected lines.
5. Send `esp32_sensor_update` to mutate state mid-run.
6. Exit 0 (pass) / 1 (fail) with a clear diagnostic block at the end.
This is the contract our **live camera test** has to satisfy. We won't
fight the pattern — we'll add `camera_frame` as a sibling of the
existing sensor messages.
## The simulation WebSocket message types we'll touch
From `backend/app/api/routes/simulation.py:90-260`:
| Message type | Direction | What it does |
|--------------------------------|-----------------|------------------------------------|
| `start_esp32` | client → server | Boots QEMU with firmware |
| `stop_esp32` | client → server | Tears QEMU down |
| `esp32_sensor_attach` | client → server | Generic sensor protocol register |
| `esp32_sensor_update` | client → server | Mutate sensor state |
| `esp32_sensor_detach` | client → server | Drop sensor |
| `esp32_uart{1,2}_input` | client → server | UART byte stream |
| `esp32_i2c_response` | client → server | I²C slave transaction reply |
| `serial_output` | server → client | Buffered UART out |
| `gpio_change` | server → client | Pin state transition |
| `system` / `error` | server → client | Lifecycle / faults |
For camera we'll add **two** new types — no need to overload the
generic sensor channel:
```
esp32_camera_attach client → server {board, jpeg_quality, frame_size}
esp32_camera_frame client → server binary JPEG payload (or base64 in JSON)
esp32_camera_detach client → server ()
```
This keeps the camera as a first-class peripheral instead of pretending
it's a "sensor", which would imply `pin` semantics that don't apply.
## Patterns to **copy verbatim** from `test_dht22_simulation.mjs`
- Session ID with timestamp: `test-camera-${Date.now()}` so parallel
runs don't collide on the backend's connection map.
- Line buffer for `serial_output` — chunks arrive partial:
```js
let _buf = '';
// … on serial_output:
_buf += data?.data ?? '';
while ((nl = _buf.indexOf('\n')) !== -1) {
const line = _buf.slice(0, nl).replace(/\r$/, '');
_buf = _buf.slice(nl + 1);
// …match expected lines…
}
```
- ANSI-coloured logging helpers (`info`, `ok`, `err`, `serial`,
`gpio`) — copy the helpers, don't re-invent them.
- Pass/fail decision at the end with concrete diagnostic guidance
("→ Check that `camera_frame` propagates to backend ring buffer").
- `--timeout=N` and `--backend=URL` CLI flags so the same script runs
in CI and locally.
## Patterns to **NOT copy**
- The DHT22 script hard-codes the sensor on GPIO4 in two places. For
the camera test we keep the pin map in **one** const block at the
top so a wiring change is one edit.
- The DHT22 script exits 1 on partial pass with a "PARTIAL" label.
The camera test doesn't have a "partial" state — either the firmware
reports back the JPEG size correctly or it doesn't. Binary pass/fail.
## Where each phase of the camera plan plugs in
```
Phase 1 (SCCB stub) → Python pytest layer (test_camera_websocket.py exists)
Phase 2 (I²S + DMA) → Python pytest layer + new C unit-tests inside QEMU
Phase 3 (frame injection) → New live e2e: test/backend/e2e/test_camera_simulation.mjs
refactor: rename wokwi-libs/ → third-party/ The directory grew well beyond Wokwi-only contents: it now hosts lcgamboa's QEMU fork (qemu-lcgamboa), Espressif's esp32-camera, the ngspice WASM build, fritzing-parts, picowi, an alternative QEMU (qemu-esp32), the 100_Days_100_IoT_Projects examples repo, and Wokwi's own avr8js/rp2040js/wokwi-elements/wokwi-features/wokwi-boards. "wokwi-libs" was misleading — half the contents have nothing to do with Wokwi. "third-party/" is the standard convention for vendored external dependencies. Mechanical changes: Path rename: wokwi-libs/ → third-party/ update-wokwi-libs.bat → update-third-party.bat docs/WOKWI_LIBS.md → docs/THIRD_PARTY.md Submodule reconfiguration: .gitmodules — 4 path= and section names updated .git/modules/wokwi-libs/ → .git/modules/third-party/ each submodule's .git file rewired to ../../.git/modules/third-party/<name> Reference updates (~80 files): vite.config.ts aliases, Dockerfile COPY paths, GH Actions workflow steps, build_qemu_*.sh, all docs/* and test/*/autosearch/* entries that mention the path, package-lock.json file: dependencies, .gitignore patterns, sitemap.xml + index.html SEO blurbs, scripts/generate-component-*, .dockerignore, .idea/vcs.xml. Bulk replaced both `wokwi-libs/` (path) and bare `wokwi-libs` (textual mentions in docs/comments). Verified: - npx tsc -b --noEmit produces no new errors related to these paths - vite.config.ts aliases now point at ../third-party/avr8js etc. - All 4 git submodules (avr8js, rp2040js, wokwi-elements, wokwi-features) are linked under third-party/ with their worktrees re-populated and config files referencing the new path - `grep -r wokwi-libs` returns zero hits outside node_modules, .vite, frontend/dist, third-party/ (upstream submodule contents), *.pyc caches, and *.dll.pre-camera rollback binaries Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 10:58:57 +07:00
Phase 4 (build) → Update third-party/qemu-lcgamboa/build_libqemu-esp32.sh
feat: ESP32-CAM emulation with webcam frame bridge First open-source end-to-end emulation of the AI-Thinker ESP32-CAM in QEMU, paired with a browser webcam → firmware bridge so users can develop camera sketches without hardware. Status: esp_camera_init() returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by the upstream driver. Final piece (cam_task accepting frames) is in progress — descriptor walker fix landed in this commit. Backend (Python/FastAPI): - simulation.py: camera_attach/frame/detach WS handlers - esp32_worker.py: ctypes binding to velxio_push_camera_frame + feature-detection fallback for older DLLs - esp32_lib_manager.py: forward camera commands to the worker stdin - esp-idf-template/main/CMakeLists.txt: esp32-camera headers added via add_prebuilt_library + REQUIRES driver (resolves i2c_master_* symbols). LED_BUILTIN=2 fallback for sketches that hardcode it. Frontend (React/TS): - EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family) added to isQemuBoard list — Run button now starts the QEMU bridge for these boards instead of falling through to the AVR path - useWebcamFrames.ts: getUserMedia → OffscreenCanvas → toBlob('image/jpeg') → base64 → WS at ~10 fps - CameraToggle.tsx: header button with status colors + frame counter - SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards - Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa - useSimulatorStore.ts: diagnostic log on compileBoardProgram - components-metadata.json: regen including esp32-cam component Submodule pointer: - wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32) Investigation + tests in test/test-esp32-cam/: - 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S spec, build blueprint, blockers resolved, descriptor walker fix) - 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip, webcam_demo) + 8 live + WS regression tests - README with the user-facing flow .gitignore: - libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated) - wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path, not part of this repo) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:28:55 +07:00
Phase 5 (frontend) → frontend/src/__tests__/esp32-camera-frame.test.ts
Phase 6 (backfill tests) → flip @expectedFailure off in tests we already wrote
```
The Phase-3 e2e is the one that proves end-to-end that the camera
works. Pseudo-code of what it'll do:
```js
// 1. compile camera_init.ino
// 2. ws.send(start_esp32 with board='esp32-cam')
// 3. ws.send(esp32_camera_attach)
// 4. ws.send(esp32_camera_frame with a 4x4 red square JPEG)
// 5. wait for serial line "got frame: <N> bytes 320x240 fmt=4"
// 6. ws.send another frame with different bytes
// 7. wait for the next serial line — bytes must differ
// 8. exit 0
```
That's the regression contract. If we ever break the frame transport,
Phase-3 e2e fails with a precise message.