velxio/test/test-esp32-cam/tests/test_camera_websocket.py

138 lines
5.5 KiB
Python
Raw Normal View History

feat: ESP32-CAM emulation with webcam frame bridge First open-source end-to-end emulation of the AI-Thinker ESP32-CAM in QEMU, paired with a browser webcam → firmware bridge so users can develop camera sketches without hardware. Status: esp_camera_init() returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by the upstream driver. Final piece (cam_task accepting frames) is in progress — descriptor walker fix landed in this commit. Backend (Python/FastAPI): - simulation.py: camera_attach/frame/detach WS handlers - esp32_worker.py: ctypes binding to velxio_push_camera_frame + feature-detection fallback for older DLLs - esp32_lib_manager.py: forward camera commands to the worker stdin - esp-idf-template/main/CMakeLists.txt: esp32-camera headers added via add_prebuilt_library + REQUIRES driver (resolves i2c_master_* symbols). LED_BUILTIN=2 fallback for sketches that hardcode it. Frontend (React/TS): - EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family) added to isQemuBoard list — Run button now starts the QEMU bridge for these boards instead of falling through to the AVR path - useWebcamFrames.ts: getUserMedia → OffscreenCanvas → toBlob('image/jpeg') → base64 → WS at ~10 fps - CameraToggle.tsx: header button with status colors + frame counter - SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards - Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa - useSimulatorStore.ts: diagnostic log on compileBoardProgram - components-metadata.json: regen including esp32-cam component Submodule pointer: - wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32) Investigation + tests in test/test-esp32-cam/: - 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S spec, build blueprint, blockers resolved, descriptor walker fix) - 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip, webcam_demo) + 8 live + WS regression tests - README with the user-facing flow .gitignore: - libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated) - wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path, not part of this repo) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 05:28:55 +07:00
"""
Layer 4 backend WS route accepts (or, today, *will* accept) the new
camera-frame message types without crashing.
The shim from autosearch/04 introduces three new WebSocket message
types:
- 'camera_frame' browser backend, binary JPEG payload
- 'esp32_camera_attach' frontend asks backend to start serving frames
- 'esp32_camera_detach' frontend tells backend to drop the queue
This layer mocks the FastAPI WebSocket and asserts the route doesn't
500/raise on those messages. It runs in-process no live backend, no
QEMU.
Today the route doesn't know the new types yet, so the relevant tests
are written as `unittest.expectedFailure` until the shim ships. That
way pytest goes green on every CI run, but a future contributor adding
the route handler gets a clear "now flip the @expectedFailure off"
signal.
"""
from __future__ import annotations
import importlib
import json
import pathlib
import sys
import unittest
from unittest.mock import AsyncMock, MagicMock, patch
# The backend is not a package on PYTHONPATH by default. Match the same
# bootstrap that test/esp32_cam/test_esp32_cam_blink.py uses so this file
# is invocable from any CWD.
_BACKEND = pathlib.Path(__file__).resolve().parent.parent.parent.parent / "backend"
if str(_BACKEND) not in sys.path:
sys.path.insert(0, str(_BACKEND))
def _make_ws(messages: list[dict]) -> MagicMock:
"""Mirror of the helper in test_esp32_cam_blink.py — gives the route a
receive_text() that yields prepared JSON messages then raises
WebSocketDisconnect, so the route's `while True` loop terminates."""
ws = MagicMock()
ws.accept = AsyncMock()
msg_iter = iter([json.dumps(m) for m in messages])
async def receive_text():
try:
return next(msg_iter)
except StopIteration:
from fastapi.websockets import WebSocketDisconnect
raise WebSocketDisconnect()
ws.receive_text = receive_text
ws.send_text = AsyncMock()
return ws
class TestCameraWebSocketMessages(unittest.IsolatedAsyncioTestCase):
"""Exercise the simulation WS route with the new message types so that:
1. **Now (no shim shipped):** the route falls through to the
"unknown message type" branch silently i.e. no exception, no
crash. That keeps the door open for the frontend to start
sending these messages early without breaking production.
2. **Once the shim ships:** flip `expectedFailure` off and assert
the route forwards the payload to `camera_queue.push()` /
`esp_lib_manager.camera_attach()`. The skeleton is here so the
diff that lands the shim is small and obvious."""
async def asyncSetUp(self):
import app.services.esp_qemu_manager as em_mod
importlib.reload(em_mod)
import app.services.esp32_lib_manager as lib_mod
importlib.reload(lib_mod)
import app.api.routes.simulation as sim_mod
importlib.reload(sim_mod)
self.sim_mod = sim_mod
self.esp = em_mod.esp_qemu_manager
self.lib = lib_mod.esp_lib_manager
async def test_camera_frame_message_does_not_crash_route(self):
"""Send a `camera_frame` JSON message; route must not raise."""
ws = _make_ws([{
"type": "camera_frame",
"data": {"fmt": "jpeg", "w": 320, "h": 240, "b64": "AAAA"},
}])
# Patch the start/stop hooks so the loop doesn't try to touch
# actual QEMU; only the message dispatch matters.
with patch.object(self.lib, "start_instance", new=AsyncMock()), \
patch.object(self.lib, "stop_instance", new=AsyncMock()), \
patch.object(self.esp, "start_instance"), \
patch.object(self.esp, "stop_instance"):
try:
await self.sim_mod.simulation_websocket(ws, "cam-frame-test")
except Exception as e: # pragma: no cover — must not happen
self.fail(f"unknown msg_type 'camera_frame' crashed route: {e}")
async def test_camera_attach_message_does_not_crash_route(self):
ws = _make_ws([{
"type": "esp32_camera_attach",
"data": {"board": "esp32-cam"},
}])
with patch.object(self.lib, "start_instance", new=AsyncMock()), \
patch.object(self.lib, "stop_instance", new=AsyncMock()), \
patch.object(self.esp, "start_instance"), \
patch.object(self.esp, "stop_instance"):
try:
await self.sim_mod.simulation_websocket(ws, "cam-attach-test")
except Exception as e: # pragma: no cover
self.fail(f"esp32_camera_attach crashed route: {e}")
@unittest.expectedFailure
async def test_camera_attach_forwards_to_lib_manager(self):
"""Once the shim lands, this should pass: 'esp32_camera_attach'
must invoke the equivalent of `esp_lib_manager.camera_attach`."""
ws = _make_ws([{
"type": "esp32_camera_attach",
"data": {"board": "esp32-cam"},
}])
with patch.object(self.lib, "camera_attach",
new=AsyncMock(), create=True) as cam_attach, \
patch.object(self.lib, "start_instance", new=AsyncMock()), \
patch.object(self.lib, "stop_instance", new=AsyncMock()), \
patch.object(self.esp, "start_instance"), \
patch.object(self.esp, "stop_instance"):
try:
await self.sim_mod.simulation_websocket(ws, "cam-fwd-test")
except Exception:
pass
cam_attach.assert_awaited()
if __name__ == "__main__":
unittest.main(verbosity=2)