velxio/test/test-esp32-cam/tests/test_sccb_probe_live.py

166 lines
5.6 KiB
Python
Raw Normal View History

feat: ESP32-CAM emulation with webcam frame bridge First open-source end-to-end emulation of the AI-Thinker ESP32-CAM in QEMU, paired with a browser webcam → firmware bridge so users can develop camera sketches without hardware. Status: esp_camera_init() returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by the upstream driver. Final piece (cam_task accepting frames) is in progress — descriptor walker fix landed in this commit. Backend (Python/FastAPI): - simulation.py: camera_attach/frame/detach WS handlers - esp32_worker.py: ctypes binding to velxio_push_camera_frame + feature-detection fallback for older DLLs - esp32_lib_manager.py: forward camera commands to the worker stdin - esp-idf-template/main/CMakeLists.txt: esp32-camera headers added via add_prebuilt_library + REQUIRES driver (resolves i2c_master_* symbols). LED_BUILTIN=2 fallback for sketches that hardcode it. Frontend (React/TS): - EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family) added to isQemuBoard list — Run button now starts the QEMU bridge for these boards instead of falling through to the AVR path - useWebcamFrames.ts: getUserMedia → OffscreenCanvas → toBlob('image/jpeg') → base64 → WS at ~10 fps - CameraToggle.tsx: header button with status colors + frame counter - SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards - Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa - useSimulatorStore.ts: diagnostic log on compileBoardProgram - components-metadata.json: regen including esp32-cam component Submodule pointer: - wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32) Investigation + tests in test/test-esp32-cam/: - 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S spec, build blueprint, blockers resolved, descriptor walker fix) - 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip, webcam_demo) + 8 live + WS regression tests - README with the user-facing flow .gitignore: - libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated) - wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path, not part of this repo) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:28:55 +07:00
"""
Phase-1 live test proves the QEMU OV2640 SCCB device answers the
chip-id dance.
Mirrors test_camera_live.py but exercises the simpler `sccb_probe.ino`
sketch, which only uses the ESP32 hardware I²C controller. No I²S, no
DMA, no DVP so a failure here unambiguously points at our SCCB
device implementation.
@unittest.expectedFailure until Phase 1 lands. Flip off in the same
PR that ships hw/i2c/esp32_ov2640.c. See
autosearch/07_ov2640_sccb_spec.md for the register table the device
must implement to make this test pass.
"""
from __future__ import annotations
import asyncio
import json
import os
import pathlib
import socket
import unittest
from urllib.parse import urlparse
_THIS_DIR = pathlib.Path(__file__).resolve().parent
_TEST_ROOT = _THIS_DIR.parent
_SKETCH = _TEST_ROOT / "sketches" / "sccb_probe" / "sccb_probe.ino"
def _backend_base_url() -> str:
return os.environ.get("VELXIO_BACKEND_URL", "").strip()
def _backend_reachable(timeout: float = 0.5) -> bool:
url = _backend_base_url()
if not url:
return False
try:
u = urlparse(url)
host = u.hostname or "localhost"
port = u.port or (443 if u.scheme == "https" else 80)
with socket.create_connection((host, port), timeout=timeout):
return True
except OSError:
return False
except Exception:
return False
def _ws_url(client_id: str) -> str:
base = _backend_base_url() or "http://localhost:8001"
u = urlparse(base)
scheme = "wss" if u.scheme == "https" else "ws"
host = u.hostname or "localhost"
port = f":{u.port}" if u.port else ""
return f"{scheme}://{host}{port}/api/simulation/ws/{client_id}"
@unittest.skipUnless(
_backend_reachable(),
"Velxio backend not reachable on $VELXIO_BACKEND_URL",
)
class TestSccbProbeLive(unittest.IsolatedAsyncioTestCase):
COMPILE_TIMEOUT = 300.0
SERIAL_TIMEOUT = 25.0
async def test_ov2640_chip_id_returned(self):
"""Compile sccb_probe.ino, boot it under QEMU, scan serial for
the four-byte chip-id signature. The exact bytes 0x26 0x42 0xa2
0x7f are spec'd in autosearch/07 — anything else means the
device misbehaves.
Phase 1 deliverable: PASSING since the OV2640 SCCB device shipped
refactor: rename wokwi-libs/ → third-party/ The directory grew well beyond Wokwi-only contents: it now hosts lcgamboa's QEMU fork (qemu-lcgamboa), Espressif's esp32-camera, the ngspice WASM build, fritzing-parts, picowi, an alternative QEMU (qemu-esp32), the 100_Days_100_IoT_Projects examples repo, and Wokwi's own avr8js/rp2040js/wokwi-elements/wokwi-features/wokwi-boards. "wokwi-libs" was misleading — half the contents have nothing to do with Wokwi. "third-party/" is the standard convention for vendored external dependencies. Mechanical changes: Path rename: wokwi-libs/ → third-party/ update-wokwi-libs.bat → update-third-party.bat docs/WOKWI_LIBS.md → docs/THIRD_PARTY.md Submodule reconfiguration: .gitmodules — 4 path= and section names updated .git/modules/wokwi-libs/ → .git/modules/third-party/ each submodule's .git file rewired to ../../.git/modules/third-party/<name> Reference updates (~80 files): vite.config.ts aliases, Dockerfile COPY paths, GH Actions workflow steps, build_qemu_*.sh, all docs/* and test/*/autosearch/* entries that mention the path, package-lock.json file: dependencies, .gitignore patterns, sitemap.xml + index.html SEO blurbs, scripts/generate-component-*, .dockerignore, .idea/vcs.xml. Bulk replaced both `wokwi-libs/` (path) and bare `wokwi-libs` (textual mentions in docs/comments). Verified: - npx tsc -b --noEmit produces no new errors related to these paths - vite.config.ts aliases now point at ../third-party/avr8js etc. - All 4 git submodules (avr8js, rp2040js, wokwi-elements, wokwi-features) are linked under third-party/ with their worktrees re-populated and config files referencing the new path - `grep -r wokwi-libs` returns zero hits outside node_modules, .vite, frontend/dist, third-party/ (upstream submodule contents), *.pyc caches, and *.dll.pre-camera rollback binaries Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 10:58:57 +07:00
in libqemu-xtensa. See third-party/qemu-lcgamboa/hw/i2c/esp32_ov2640.c."""
feat: ESP32-CAM emulation with webcam frame bridge First open-source end-to-end emulation of the AI-Thinker ESP32-CAM in QEMU, paired with a browser webcam → firmware bridge so users can develop camera sketches without hardware. Status: esp_camera_init() returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by the upstream driver. Final piece (cam_task accepting frames) is in progress — descriptor walker fix landed in this commit. Backend (Python/FastAPI): - simulation.py: camera_attach/frame/detach WS handlers - esp32_worker.py: ctypes binding to velxio_push_camera_frame + feature-detection fallback for older DLLs - esp32_lib_manager.py: forward camera commands to the worker stdin - esp-idf-template/main/CMakeLists.txt: esp32-camera headers added via add_prebuilt_library + REQUIRES driver (resolves i2c_master_* symbols). LED_BUILTIN=2 fallback for sketches that hardcode it. Frontend (React/TS): - EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family) added to isQemuBoard list — Run button now starts the QEMU bridge for these boards instead of falling through to the AVR path - useWebcamFrames.ts: getUserMedia → OffscreenCanvas → toBlob('image/jpeg') → base64 → WS at ~10 fps - CameraToggle.tsx: header button with status colors + frame counter - SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards - Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa - useSimulatorStore.ts: diagnostic log on compileBoardProgram - components-metadata.json: regen including esp32-cam component Submodule pointer: - wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32) Investigation + tests in test/test-esp32-cam/: - 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S spec, build blueprint, blockers resolved, descriptor walker fix) - 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip, webcam_demo) + 8 live + WS regression tests - README with the user-facing flow .gitignore: - libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated) - wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path, not part of this repo) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:28:55 +07:00
try:
import httpx # type: ignore
import websockets # type: ignore
except ImportError as exc:
self.skipTest(f"missing test deps: {exc}")
sketch = _SKETCH.read_text(encoding="utf-8")
sketch_name = _SKETCH.name
async with httpx.AsyncClient(
base_url=_backend_base_url(), timeout=self.COMPILE_TIMEOUT,
) as http:
res = await http.post("/api/compile/", json={
"files": [{"name": sketch_name, "content": sketch}],
"board_fqbn": "esp32:esp32:esp32cam",
})
self.assertEqual(
res.status_code, 200,
f"/api/compile/ HTTP {res.status_code}: {res.text[:400]}",
)
body = res.json()
if not body.get("success"):
self.skipTest(
f"compile failure: "
f"{(body.get('error') or body.get('stderr', ''))[:600]}"
)
firmware_b64 = body.get("binary_content") or body.get("firmware_b64")
self.assertTrue(firmware_b64)
client_id = (
f"esp32-cam-sccb-test-"
f"{int(asyncio.get_event_loop().time() * 1000)}"
)
url = _ws_url(client_id)
async with websockets.connect(
url, ping_interval=None, max_size=4 * 1024 * 1024,
) as ws:
await ws.send(json.dumps({
"type": "start_esp32",
"data": {"board": "esp32-cam", "firmware_b64": firmware_b64},
}))
saw_chip_id = False
saw_detected = False
buf = ""
try:
deadline = asyncio.get_event_loop().time() + self.SERIAL_TIMEOUT
while asyncio.get_event_loop().time() < deadline:
remaining = deadline - asyncio.get_event_loop().time()
if remaining <= 0:
break
raw = await asyncio.wait_for(ws.recv(), timeout=remaining)
try:
msg = json.loads(raw)
except json.JSONDecodeError:
continue
if msg.get("type") == "serial_output":
buf += msg.get("data", {}).get("data", "")
if "PID=0x26 VER=0x42 MIDH=0xA2 MIDL=0x7F" in buf:
saw_chip_id = True
if "OV2640 detected" in buf:
saw_detected = True
if saw_chip_id and saw_detected:
break
except asyncio.TimeoutError:
pass
finally:
try:
await ws.send(json.dumps({"type": "stop_esp32", "data": {}}))
except Exception:
pass
self.assertTrue(
saw_chip_id,
f"firmware never printed the OV2640 chip-id (looked for "
f"'PID=0x26 VER=0x42 MIDH=0xA2 MIDL=0x7F'). Buffered serial: "
f"{buf[:400]}",
)
self.assertTrue(
saw_detected,
"firmware printed wrong chip-id bytes — device emulation "
"returns the wrong values",
)
if __name__ == "__main__":
unittest.main(verbosity=2)