184 lines
7.5 KiB
Markdown
184 lines
7.5 KiB
Markdown
# 08 — DVP + I²S parallel-input spec for ESP32
|
||
|
||
The hardest part of the emulation. The ESP32 has no dedicated camera
|
||
peripheral — it reuses **I²S0 in slave-RX mode** to capture an 8-bit
|
||
parallel video stream, and routes the data into RAM via the GDMA
|
||
linked-list mechanism.
|
||
|
||
Sources for this note:
|
||
|
||
- ESP32 TRM (2023 rev) chapter 12 (I²S Controller) §12.4.6 (LCD/Cam mode).
|
||
- `espressif/esp32-camera/target/esp32/ll_cam.c` — start/stop/IRQ.
|
||
- `espressif/esp32-camera/driver/cam_hal.c` — DMA descriptor walking.
|
||
|
||
## I²S0 register block
|
||
|
||
- Base: **`DR_REG_I2S_BASE = 0x3FF4F000`**.
|
||
- 4 KB MMIO region.
|
||
- Currently mapped as **unimplemented** in our QEMU fork
|
||
(`hw/xtensa/esp32_picsimlab.c:750`):
|
||
|
||
```c
|
||
esp32_soc_add_unimp_device(sys_mem, "esp32.i2s0", DR_REG_I2S_BASE, 0x1000, 0);
|
||
esp32_soc_add_unimp_device(sys_mem, "esp32.i2s1", DR_REG_I2S1_BASE, 0x1000, 0);
|
||
```
|
||
|
||
Phase-2 deliverable: replace this with a real `esp32_i2s_cam` device
|
||
for I²S0; leave I²S1 unimplemented (audio not in scope).
|
||
|
||
## Register start sequence (driver — `ll_cam_start`)
|
||
|
||
Quoting `target/esp32/ll_cam.c` verbatim:
|
||
|
||
```c
|
||
I2S0.conf.rx_start = 0; // stop any in-flight RX
|
||
I2S_ISR_ENABLE(in_suc_eof); // arm EOF interrupt
|
||
I2S0.conf.rx_reset = 1;
|
||
I2S0.conf.rx_reset = 0;
|
||
I2S0.conf.rx_fifo_reset = 1;
|
||
I2S0.conf.rx_fifo_reset = 0;
|
||
I2S0.lc_conf.in_rst = 1;
|
||
I2S0.lc_conf.in_rst = 0;
|
||
I2S0.lc_conf.ahbm_fifo_rst = 1;
|
||
I2S0.lc_conf.ahbm_fifo_rst = 0;
|
||
I2S0.lc_conf.ahbm_rst = 1;
|
||
I2S0.lc_conf.ahbm_rst = 0;
|
||
I2S0.rx_eof_num = cam->dma_half_buffer_size / sizeof(dma_elem_t);
|
||
I2S0.in_link.addr = ((uint32_t)&cam->dma[0]) & 0xfffff;
|
||
I2S0.in_link.start = 1;
|
||
I2S0.conf.rx_start = 1;
|
||
```
|
||
|
||
The **registers our QEMU model must implement** (offsets from
|
||
`DR_REG_I2S_BASE`):
|
||
|
||
| Offset | Name | What we do |
|
||
|--------|------------------|-----------------------------------------------|
|
||
| 0x008 | `conf` | watch `rx_start`, `rx_reset`, `rx_fifo_reset` |
|
||
| 0x01C | `int_raw` | RW1C, where `in_suc_eof` lives |
|
||
| 0x020 | `int_st` | masked status |
|
||
| 0x024 | `int_ena` | mask |
|
||
| 0x028 | `int_clr` | RW1C clear |
|
||
| 0x028 | `lc_conf` | watch `in_rst`, `ahbm_fifo_rst`, `ahbm_rst` |
|
||
| 0x024 | `rx_eof_num` | EOF threshold in samples |
|
||
| 0x028 | `in_link` | DMA descriptor head + start bit |
|
||
|
||
(The exact offsets above are approximate; final mapping per the
|
||
`I2S_*_REG` macros in `soc/i2s_struct.h`. The Phase-2 PR derives them
|
||
from `third-party/qemu-lcgamboa`'s existing field-AP scheme — see
|
||
`hw/i2c/esp32_i2c.c` for the pattern.)
|
||
|
||
## Linked-list DMA descriptor (`lldesc_t`)
|
||
|
||
12 bytes, big-endian-on-little-endian-arch packed struct, lives in
|
||
`soc/lldesc.h`:
|
||
|
||
```c
|
||
typedef struct lldesc {
|
||
volatile uint32_t size : 12; // physical buffer size
|
||
volatile uint32_t length : 12; // bytes to transfer
|
||
volatile uint32_t offset : 5; // unused for camera RX
|
||
volatile uint32_t sosf : 1; // start-of-sub-frame
|
||
volatile uint32_t eof : 1; // end-of-frame
|
||
volatile uint32_t owner : 1; // 1 = HW, 0 = CPU
|
||
volatile uint32_t buf_ptr; // physical buffer address
|
||
volatile uint32_t next; // next descriptor or NULL
|
||
} lldesc_t;
|
||
```
|
||
|
||
Our QEMU model walks the linked list as data arrives:
|
||
|
||
1. Read descriptor at `in_link.addr`.
|
||
2. While `descr.owner == 1` and there's pending frame data:
|
||
- Copy up to `descr.size` bytes (with `dma_elem_t` packing — see
|
||
below) into `descr.buf_ptr`.
|
||
- Decrement remaining; advance `descr.next`.
|
||
3. When the configured EOF condition is hit
|
||
(`bytes_written / sizeof(dma_elem_t) == rx_eof_num`):
|
||
- Set `descr.eof = 1`, `descr.owner = 0` (give back to CPU).
|
||
- Raise `int_raw.in_suc_eof = 1`.
|
||
- If `int_ena.in_suc_eof = 1`, fire IRQ to CPU.
|
||
|
||
## `dma_elem_t` — the 8-bit-data padding format
|
||
|
||
Because I²S is fundamentally a 16-bit-aligned bus, the camera driver
|
||
configures it to capture **8 bits of real data + 8 bits of padding
|
||
per sample**. The driver-side type is:
|
||
|
||
```c
|
||
typedef union {
|
||
struct {
|
||
uint32_t sample2 : 8; // second pixel-byte
|
||
uint32_t unused2 : 8;
|
||
uint32_t sample1 : 8; // first pixel-byte
|
||
uint32_t unused1 : 8;
|
||
};
|
||
uint32_t val;
|
||
} dma_elem_t;
|
||
```
|
||
|
||
So **for every 4 bytes our QEMU device writes to RAM, only 2 bytes
|
||
carry image data**. The driver later re-packs these via the
|
||
`ll_cam_dma_filter_*` callbacks before exposing the buffer to the
|
||
user.
|
||
|
||
Implementation rule for the QEMU model: when the host pushes a
|
||
**32-byte run of pixel bytes**, write **64 bytes** to the descriptor
|
||
buffer (each pixel byte zero-padded into a 16-bit slot, two slots per
|
||
32-bit word, in `sample1 / sample2` order).
|
||
|
||
This is the single trickiest detail of the emulation. The Phase-2 unit
|
||
test (in C, as a QEMU device test) MUST cover this padding explicitly.
|
||
|
||
## Frame timing — what we synthesise
|
||
|
||
Real OV2640 at QVGA-JPEG ~10 fps with PCLK=10MHz produces:
|
||
|
||
- `VSYNC` once per 100 ms (HIGH ~1.5 ms).
|
||
- `HREF` per line (~240 lines), HIGH ~640 µs.
|
||
- 8-bit byte every PCLK rising edge while HREF is HIGH.
|
||
- Frame size is **variable** (JPEG): typically 8–14 KB at QVGA Q60.
|
||
|
||
For QEMU we don't need to be cycle-accurate. Two strategies:
|
||
|
||
- **(A)** Drop the entire frame into RAM in one go when the driver
|
||
starts a capture. Set EOF after the right number of `dma_elem_t`s.
|
||
Simulate `VSYNC` by raising `in_suc_eof` after the write.
|
||
- **(B)** Pace the writes via a `qemu_mod_timer` that fires every
|
||
~100 µs, writing `~64 bytes`. More authentic, but slower.
|
||
|
||
**Phase 2 ships strategy (A).** Strategy (B) only matters for sketches
|
||
that want to measure VSYNC timing — extremely rare. Document the
|
||
limitation in `05_open_questions.md`.
|
||
|
||
## Pixel-format conversion
|
||
|
||
`camera_config_t.pixel_format`:
|
||
|
||
| value | What's in the buffer after `fb_get` |
|
||
|----------------|--------------------------------------------------|
|
||
| PIXFORMAT_JPEG | JPEG bytes (variable length) |
|
||
| PIXFORMAT_RGB565 | width*height*2 bytes, RGB565 |
|
||
| PIXFORMAT_YUV422 | width*height*2 bytes |
|
||
| PIXFORMAT_GRAYSCALE | width*height bytes |
|
||
|
||
Browser webcam → host-side conversion table:
|
||
|
||
| Frontend produces | Backend stores in queue | When firmware wants… |
|
||
|-------------------|-------------------------|----------------------|
|
||
| JPEG (canvas.toBlob) | JPEG bytes | JPEG → pass-through |
|
||
| | | RGB565 → decode + pack |
|
||
| | | GRAY → decode + Y plane |
|
||
|
||
The decoding burden lives in the backend (`esp_lib_bridge`), not in
|
||
QEMU. QEMU just pushes "the bytes for this frame" — what's in the
|
||
bytes is whatever the host already converted.
|
||
|
||
## Sources
|
||
|
||
- [esp32-camera/target/esp32/ll_cam.c](https://github.com/espressif/esp32-camera/blob/master/target/esp32/ll_cam.c)
|
||
- [esp32-camera/driver/cam_hal.c](https://github.com/espressif/esp32-camera/blob/master/driver/cam_hal.c)
|
||
- [I2S camera mode driver issues — IDFGH-2582](https://github.com/espressif/esp-idf/issues/2251)
|
||
- [ESP32 forum — 8-bit parallel capture using I2S+DMA](https://www.esp32.com/viewtopic.php?t=5873)
|
||
- [DeepWiki — ESP32 Camera Driver overview](https://deepwiki.com/espressif/esp32-camera/4.3-esp32-camera-driver)
|