2026-04-05 00:27:52 +07:00
|
|
|
# ---- Stage 0: QEMU .so + ROM binaries ----
|
2026-04-07 13:58:52 +07:00
|
|
|
# Downloads arch-specific .so from GitHub Release (e.g. libqemu-xtensa-amd64.so)
|
|
|
|
|
# and renames to libqemu-xtensa.so so the backend needs no changes.
|
|
|
|
|
# Local prebuilt files (prebuilt/qemu/) are used if present.
|
2026-04-05 00:27:52 +07:00
|
|
|
FROM ubuntu:22.04 AS qemu-provider
|
|
|
|
|
|
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates \
|
2026-03-15 00:35:59 +07:00
|
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
|
|
2026-04-07 13:58:52 +07:00
|
|
|
ARG TARGETARCH
|
2026-04-05 00:27:52 +07:00
|
|
|
ARG QEMU_RELEASE_URL=https://github.com/davidmonterocrespo24/velxio/releases/download/qemu-prebuilt
|
2026-03-15 00:35:59 +07:00
|
|
|
|
2026-04-05 00:27:52 +07:00
|
|
|
# Copy the prebuilt directory (may contain .so+ROM files or just the .gitkeep)
|
|
|
|
|
RUN mkdir -p /qemu
|
|
|
|
|
COPY prebuilt/qemu/ /qemu/
|
2026-03-15 00:35:59 +07:00
|
|
|
|
2026-04-07 13:58:52 +07:00
|
|
|
# Download arch-specific .so and arch-independent ROM files
|
2026-04-05 00:27:52 +07:00
|
|
|
RUN cd /qemu \
|
2026-04-07 13:58:52 +07:00
|
|
|
&& for base in libqemu-xtensa libqemu-riscv32; do \
|
|
|
|
|
f="${base}.so" ; \
|
2026-04-05 00:27:52 +07:00
|
|
|
if [ ! -f "$f" ]; then \
|
2026-04-07 13:58:52 +07:00
|
|
|
echo "Downloading ${base}-${TARGETARCH}.so → $f ..." ; \
|
|
|
|
|
curl -fSL -o "$f" "${QEMU_RELEASE_URL}/${base}-${TARGETARCH}.so" ; \
|
|
|
|
|
else \
|
|
|
|
|
echo "Using local $f ($(stat -c%s "$f") bytes)" ; \
|
|
|
|
|
fi ; \
|
|
|
|
|
done \
|
|
|
|
|
&& for f in esp32-v3-rom.bin esp32-v3-rom-app.bin esp32c3-rom.bin; do \
|
|
|
|
|
if [ ! -f "$f" ]; then \
|
|
|
|
|
echo "Downloading $f ..." ; \
|
2026-04-05 00:27:52 +07:00
|
|
|
curl -fSL -o "$f" "${QEMU_RELEASE_URL}/$f" ; \
|
|
|
|
|
else \
|
|
|
|
|
echo "Using local $f ($(stat -c%s "$f") bytes)" ; \
|
|
|
|
|
fi ; \
|
|
|
|
|
done \
|
|
|
|
|
&& ls -lh /qemu/
|
2026-03-15 00:35:59 +07:00
|
|
|
|
|
|
|
|
|
2026-04-01 06:53:56 +07:00
|
|
|
# ---- Stage 0.5: ESP-IDF toolchain for ESP32 compilation ----
|
|
|
|
|
FROM ubuntu:22.04 AS espidf-builder
|
|
|
|
|
|
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
|
|
|
git wget flex bison gperf python3 python3-pip python3-venv \
|
|
|
|
|
cmake ninja-build ccache libffi-dev libssl-dev \
|
|
|
|
|
libusb-1.0-0 ca-certificates \
|
|
|
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
|
|
|
|
|
|
# Install ESP-IDF 4.4.7 (matches Arduino ESP32 core 2.0.17 / lcgamboa QEMU ROM)
|
|
|
|
|
RUN git clone -b v4.4.7 --recursive --depth=1 --shallow-submodules \
|
|
|
|
|
https://github.com/espressif/esp-idf.git /opt/esp-idf
|
|
|
|
|
|
|
|
|
|
WORKDIR /opt/esp-idf
|
|
|
|
|
|
|
|
|
|
# Install toolchains for esp32 (Xtensa) and esp32c3 (RISC-V) only
|
|
|
|
|
RUN ./install.sh esp32,esp32c3
|
|
|
|
|
|
|
|
|
|
# Clean up large unnecessary files to reduce image size
|
|
|
|
|
RUN rm -rf .git docs examples \
|
|
|
|
|
&& find /root/.espressif -name '*.tar.*' -delete 2>/dev/null || true
|
|
|
|
|
|
2026-04-02 08:41:52 +07:00
|
|
|
# Install Arduino-as-component for full Arduino API support in ESP-IDF builds
|
|
|
|
|
RUN git clone --branch 2.0.17 --depth=1 --recursive --shallow-submodules \
|
|
|
|
|
https://github.com/espressif/arduino-esp32.git /opt/arduino-esp32 \
|
|
|
|
|
&& rm -rf /opt/arduino-esp32/.git
|
|
|
|
|
|
2026-04-01 06:53:56 +07:00
|
|
|
|
refactor: rename wokwi-libs/ → third-party/
The directory grew well beyond Wokwi-only contents: it now hosts
lcgamboa's QEMU fork (qemu-lcgamboa), Espressif's esp32-camera, the
ngspice WASM build, fritzing-parts, picowi, an alternative QEMU
(qemu-esp32), the 100_Days_100_IoT_Projects examples repo, and
Wokwi's own avr8js/rp2040js/wokwi-elements/wokwi-features/wokwi-boards.
"wokwi-libs" was misleading — half the contents have nothing to do
with Wokwi. "third-party/" is the standard convention for vendored
external dependencies.
Mechanical changes:
Path rename:
wokwi-libs/ → third-party/
update-wokwi-libs.bat → update-third-party.bat
docs/WOKWI_LIBS.md → docs/THIRD_PARTY.md
Submodule reconfiguration:
.gitmodules — 4 path= and section names updated
.git/modules/wokwi-libs/ → .git/modules/third-party/
each submodule's .git file rewired to ../../.git/modules/third-party/<name>
Reference updates (~80 files): vite.config.ts aliases, Dockerfile
COPY paths, GH Actions workflow steps, build_qemu_*.sh, all
docs/* and test/*/autosearch/* entries that mention the path,
package-lock.json file: dependencies, .gitignore patterns,
sitemap.xml + index.html SEO blurbs, scripts/generate-component-*,
.dockerignore, .idea/vcs.xml. Bulk replaced both `wokwi-libs/`
(path) and bare `wokwi-libs` (textual mentions in docs/comments).
Verified:
- npx tsc -b --noEmit produces no new errors related to these paths
- vite.config.ts aliases now point at ../third-party/avr8js etc.
- All 4 git submodules (avr8js, rp2040js, wokwi-elements,
wokwi-features) are linked under third-party/ with their
worktrees re-populated and config files referencing the new path
- `grep -r wokwi-libs` returns zero hits outside node_modules,
.vite, frontend/dist, third-party/ (upstream submodule contents),
*.pyc caches, and *.dll.pre-camera rollback binaries
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 10:58:57 +07:00
|
|
|
# ---- Stage 1: Build frontend and third-party ----
|
2026-03-06 22:09:55 +07:00
|
|
|
FROM node:20 AS frontend-builder
|
2026-03-06 20:14:50 +07:00
|
|
|
|
|
|
|
|
WORKDIR /app
|
|
|
|
|
|
fix(install): unblock self-hosting + drop forced wokwi clones
Resolves several install pain points reported by users (#108, #120) and
removes the obligatory upstream-clone step that confused contributors and
slowed down every Docker build.
Install fixes:
- nginx: server_name → catch-all default_server, drop Debian's stock site
so reverse-proxied users no longer get the "Welcome to nginx" page.
- entrypoint: auto-generate SECRET_KEY at first boot, persisted under
data/.secret_key. backend/.env is now optional in docker-compose.yml.
- backend: add greenlet>=3.0.0 (SQLAlchemy async dep that was missing on
some Python builds — caused uvicorn startup failures on WSL).
Wokwi libs come from npm:
- @wokwi/elements 1.9.2, avr8js 0.21.0, rp2040js 1.3.2 are pinned in
frontend/package.json. Vite aliases removed.
- Dockerfile.standalone no longer clones avr8js / rp2040js / wokwi-elements
/ wokwi-boards. Frontend stage is just COPY + npm install + build:docker.
- Board SVGs vendored under frontend/public/boards/ (10 deduped against
existing files, 2 truly new). third-party/wokwi-* clones become reference-
only credits — generate-component-metadata.ts skips gracefully when absent.
Production config split out:
- docker-compose.prod.yml, deploy/nginx.prod.conf, nginx-host-velxio*.conf,
update-third-party.bat removed. Production deployment lives in its own
repo: https://github.com/velxio/velxio-prod (host nginx + HTTPS + backups
+ pinned upstream commit).
Verified locally: 1161 frontend tests pass, build:docker completes clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 10:04:11 +07:00
|
|
|
# avr8js, rp2040js and @wokwi/elements are pulled directly from the npm
|
|
|
|
|
# registry (see frontend/package.json) — no upstream git clones needed.
|
|
|
|
|
# Board SVGs live in frontend/public/boards/, component SVGs in
|
|
|
|
|
# frontend/public/component-svgs/, and components-metadata.json is committed.
|
2026-03-06 20:14:50 +07:00
|
|
|
COPY frontend/ frontend/
|
2026-04-15 03:38:03 +07:00
|
|
|
COPY scripts/ scripts/
|
2026-03-06 20:14:50 +07:00
|
|
|
WORKDIR /app/frontend
|
2026-05-04 11:08:23 +07:00
|
|
|
# Lock files aren't committed in this repo (they're gitignored) — see the
|
|
|
|
|
# note in .gitignore. The `rm -f` below is defense-in-depth in case
|
|
|
|
|
# someone runs `docker build .` from a tree where a local lock exists.
|
2026-05-04 11:04:32 +07:00
|
|
|
RUN rm -f package-lock.json \
|
|
|
|
|
&& npm install --include=optional \
|
|
|
|
|
&& npm run build:docker
|
2026-03-06 20:14:50 +07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
# ---- Stage 2: Final Production Image ----
|
|
|
|
|
FROM python:3.12-slim
|
|
|
|
|
|
2026-03-15 00:35:59 +07:00
|
|
|
# Install system dependencies, nginx, and QEMU .so runtime libraries
|
2026-03-06 20:14:50 +07:00
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
|
|
|
curl \
|
|
|
|
|
ca-certificates \
|
|
|
|
|
nginx \
|
2026-03-15 00:35:59 +07:00
|
|
|
libglib2.0-0 \
|
|
|
|
|
libgcrypt20 \
|
|
|
|
|
libslirp0 \
|
|
|
|
|
libpixman-1-0 \
|
2026-03-25 03:28:56 +07:00
|
|
|
libfdt1 \
|
2026-04-01 12:28:01 +07:00
|
|
|
cmake \
|
|
|
|
|
ninja-build \
|
|
|
|
|
libusb-1.0-0 \
|
2026-04-01 12:39:46 +07:00
|
|
|
git \
|
perf(espidf): drop in ccache for ESP32 compiles (~10× warm speedup)
Cold first compile per container is unchanged (cache empty). Subsequent
compiles drop from ~5-7 minutes to ~30-60 seconds because every ESP-IDF
base object (FreeRTOS, lwIP, esp_wifi, libsodium, soc, hal, …) hits the
cache. The user's BMP280 example, which hangs on cold compile, completes
near-instantly on the second attempt.
Why a transparent cache is safe: ccache hashes the preprocessed source +
flags + compiler. A cache hit only happens when the input is byte-for-byte
identical to a prior compile. Different sketches with different libraries
still get correct cache misses; there is no path where one project's
output contaminates another.
Changes
- Dockerfile.standalone: install ccache, set CCACHE_DIR=/var/cache/ccache,
IDF_CCACHE_ENABLE=1, configure 2 GB cap with compression. Compression
(level 6) cuts cache disk usage by ~40% with negligible CPU overhead.
- docker-compose.yml: named volume `ccache:/var/cache/ccache` so the
cache survives `docker compose up -d --build` (without it, every image
rebuild discards the cache).
- backend/app/services/espidf_compiler.py: pass `-DCCACHE_ENABLE=1` to
cmake when IDF_CCACHE_ENABLE is truthy. ESP-IDF's project.cmake
(`set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)` on line 374)
is what actually wires ccache in; without the cmake -D flag the env
var alone has no effect because we don't go through idf.py.
Escape hatch: set IDF_CCACHE_ENABLE=0 in compose env to disable without
rebuilding the image.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 10:44:29 +07:00
|
|
|
ccache \
|
2026-03-06 20:14:50 +07:00
|
|
|
&& apt-get clean \
|
2026-04-01 12:39:46 +07:00
|
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
|
|
|
&& pip install --no-cache-dir packaging
|
2026-03-06 20:14:50 +07:00
|
|
|
|
2026-03-06 21:20:47 +07:00
|
|
|
# Install arduino-cli into /usr/local/bin directly (avoids touching /bin)
|
|
|
|
|
RUN curl -fsSL https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh \
|
|
|
|
|
| BINDIR=/usr/local/bin sh
|
|
|
|
|
|
2026-03-25 04:20:38 +07:00
|
|
|
# Only install arduino-cli binary here. Core installation (arduino:avr,
|
2026-04-01 06:53:56 +07:00
|
|
|
# rp2040:rp2040) is done at first boot by entrypoint.sh and persisted
|
|
|
|
|
# in the mounted /root/.arduino15 volume.
|
|
|
|
|
# ESP32 compilation uses ESP-IDF instead of arduino-cli.
|
2026-03-06 20:14:50 +07:00
|
|
|
|
|
|
|
|
WORKDIR /app
|
|
|
|
|
|
2026-03-07 06:32:36 +07:00
|
|
|
# Data directory for persistent SQLite database (mounted as a volume at runtime)
|
|
|
|
|
RUN mkdir -p /app/data
|
|
|
|
|
|
2026-03-06 20:14:50 +07:00
|
|
|
# Install Python backend dependencies
|
|
|
|
|
COPY backend/requirements.txt .
|
|
|
|
|
RUN pip install --no-cache-dir -r requirements.txt
|
|
|
|
|
|
|
|
|
|
# Copy backend application code
|
|
|
|
|
COPY backend/app/ ./app/
|
|
|
|
|
|
feat: persist multi-board projects + add auto-save
The project save/load pipeline only persisted a single `board_type`, so
multi-board workspaces silently lost every board except the active one
on save, and wires referencing the dropped boards' IDs orphaned to the
canvas corner on reload. An audit of the production backup found 74/306
projects (24%) with at least one orphaned wire and 174/301 non-trivial
projects whose code was still the default Blink template — strong signal
that users save once and never re-save.
Backend
- Add `boards_json` column on `projects` with idempotent ALTER TABLE in
the lifespan migration list.
- New `FileGroup` schema + `file_groups` array on
ProjectCreate/Update/Response. Legacy `files`/`code` kept for back-compat.
- `project_files.py` now uses `{pid}/{groupId}/{filename}` subdirs via
`read_groups`/`write_groups`. Legacy flat layouts are auto-promoted on
read; legacy single-list `files` only updates the active group, leaving
other boards' files intact.
- `_persist_files_from_body` honors file_groups → files → code priority.
Frontend
- `useSimulatorStore.addBoard` accepts an optional `explicitId` so
saved board IDs can be restored verbatim (wires reference IDs literally).
- New `loadProjectState({boards, fileGroups, components, wires,
activeBoardId})` action: tears down current boards, recreates from the
payload, restores file groups atomically, recalculates wire positions
on the next frame, and refreshes the Interconnect.
- `useEditorStore.replaceFileGroups` for atomic multi-group restore.
- `SaveProjectModal` and `ProjectByIdPage`/`ProjectPage` now go through
`buildSavePayload` / `buildLoadPayload` (handles pre-backfill projects
by synthesising a default board from `board_type`).
Auto-save (#useAutoSaveProject hook)
- 2.5s debounced silent PUT triggered ONLY when an authenticated user
has a `currentProject` with a UUID. State hash detects real changes
vs. UI-only churn; baseline is reset on project load so the just-loaded
state isn't immediately re-saved.
- `beforeunload` flush via `fetch keepalive: true` (supports PUT +
credentials, survives unload).
- Compact status indicator in `AppHeader` (idle/dirty/saving/saved/error).
Backfill script (one-off, idempotent)
- `backend/scripts/backfill_boards_2026_05.py` populates `boards_json`
for legacy projects. Heuristic per project, based on which board IDs
the wires reference:
Case A — wires only ref 'arduino-uno' but board_type ≠ uno:
rename id→board_type and rewrite wire endpoints.
Case B — single-board normal: keep verbatim.
Case C — multi-board: recreate one board per distinct ref, infer
kind by stripping trailing -N suffix.
Also moves any flat files into the active board's group subdir.
Stdlib-only, runs from host or `docker exec`.
Docker
- `Dockerfile.standalone` now copies `backend/scripts/` into the image
so the backfill is callable via `docker exec velxio-app python
/app/scripts/backfill_boards_2026_05.py --apply`.
Verified locally on the restored production backup (363 projects):
33 Case A, 316 Case B, 14 Case C, 135 wire endpoints renamed, 0 orphans.
Re-running the script after apply skips all 363 (idempotent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 23:43:33 +07:00
|
|
|
# One-off maintenance scripts (e.g. backfill_boards_2026_05). Pure stdlib —
|
|
|
|
|
# run with: docker exec velxio-app python /app/scripts/<script> --apply
|
|
|
|
|
COPY backend/scripts/ ./scripts/
|
|
|
|
|
|
fix(install): unblock self-hosting + drop forced wokwi clones
Resolves several install pain points reported by users (#108, #120) and
removes the obligatory upstream-clone step that confused contributors and
slowed down every Docker build.
Install fixes:
- nginx: server_name → catch-all default_server, drop Debian's stock site
so reverse-proxied users no longer get the "Welcome to nginx" page.
- entrypoint: auto-generate SECRET_KEY at first boot, persisted under
data/.secret_key. backend/.env is now optional in docker-compose.yml.
- backend: add greenlet>=3.0.0 (SQLAlchemy async dep that was missing on
some Python builds — caused uvicorn startup failures on WSL).
Wokwi libs come from npm:
- @wokwi/elements 1.9.2, avr8js 0.21.0, rp2040js 1.3.2 are pinned in
frontend/package.json. Vite aliases removed.
- Dockerfile.standalone no longer clones avr8js / rp2040js / wokwi-elements
/ wokwi-boards. Frontend stage is just COPY + npm install + build:docker.
- Board SVGs vendored under frontend/public/boards/ (10 deduped against
existing files, 2 truly new). third-party/wokwi-* clones become reference-
only credits — generate-component-metadata.ts skips gracefully when absent.
Production config split out:
- docker-compose.prod.yml, deploy/nginx.prod.conf, nginx-host-velxio*.conf,
update-third-party.bat removed. Production deployment lives in its own
repo: https://github.com/velxio/velxio-prod (host nginx + HTTPS + backups
+ pinned upstream commit).
Verified locally: 1161 frontend tests pass, build:docker completes clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 10:04:11 +07:00
|
|
|
# Setup Nginx configuration. Remove Debian's stock site so it doesn't shadow
|
|
|
|
|
# ours as the default_server (was Issue #108: users behind reverse proxies got
|
|
|
|
|
# the "Welcome to nginx" page because the stock site claimed default_server).
|
|
|
|
|
RUN rm -f /etc/nginx/sites-enabled/default
|
2026-05-04 10:05:55 +07:00
|
|
|
COPY docker/nginx.conf /etc/nginx/conf.d/default.conf
|
2026-03-06 20:14:50 +07:00
|
|
|
|
|
|
|
|
# Copy built frontend assets from builder stage
|
|
|
|
|
COPY --from=frontend-builder /app/frontend/dist /usr/share/nginx/html
|
|
|
|
|
|
2026-04-05 04:02:48 +07:00
|
|
|
# Copy and configure entrypoint script (fix Windows CRLF → LF)
|
2026-05-04 10:05:55 +07:00
|
|
|
COPY docker/entrypoint.sh /app/entrypoint.sh
|
2026-04-05 04:02:48 +07:00
|
|
|
RUN sed -i 's/\r$//' /app/entrypoint.sh && chmod +x /app/entrypoint.sh
|
2026-03-06 20:14:50 +07:00
|
|
|
|
2026-04-05 00:27:52 +07:00
|
|
|
# ── ESP32 emulation: pre-built QEMU .so + ROM binaries ──────────────────────
|
|
|
|
|
# Downloaded from GitHub Release (public — no access to qemu-lcgamboa needed)
|
2026-03-19 09:30:45 +07:00
|
|
|
# libqemu-xtensa.so → ESP32 / ESP32-S3 (Xtensa LX6/LX7)
|
|
|
|
|
# libqemu-riscv32.so → ESP32-C3 (RISC-V RV32IMC)
|
|
|
|
|
# esp32-v3-rom*.bin → boot/app ROM images required by esp32-picsimlab machine
|
|
|
|
|
# esp32c3-rom.bin → ROM image required by esp32c3-picsimlab machine
|
|
|
|
|
# NOTE: ROM files must live in the same directory as the .so (worker passes -L
|
|
|
|
|
# to QEMU pointing at os.path.dirname(lib_path))
|
2026-03-15 00:35:59 +07:00
|
|
|
RUN mkdir -p /app/lib
|
2026-04-05 00:27:52 +07:00
|
|
|
COPY --from=qemu-provider /qemu/ /app/lib/
|
2026-03-15 00:35:59 +07:00
|
|
|
|
2026-03-19 09:30:45 +07:00
|
|
|
# Activate ESP32 emulation
|
|
|
|
|
# QEMU_ESP32_LIB → Xtensa library (ESP32, ESP32-S3)
|
|
|
|
|
# QEMU_RISCV32_LIB → RISC-V library (ESP32-C3 and variants)
|
2026-03-15 00:35:59 +07:00
|
|
|
ENV QEMU_ESP32_LIB=/app/lib/libqemu-xtensa.so
|
2026-03-19 09:30:45 +07:00
|
|
|
ENV QEMU_RISCV32_LIB=/app/lib/libqemu-riscv32.so
|
2026-03-15 00:35:59 +07:00
|
|
|
|
2026-04-01 06:53:56 +07:00
|
|
|
# ── ESP-IDF toolchain for ESP32 compilation ──────────────────────────────────
|
|
|
|
|
# Copied from espidf-builder stage: IDF framework + cross-compiler toolchains
|
|
|
|
|
COPY --from=espidf-builder /opt/esp-idf /opt/esp-idf
|
|
|
|
|
COPY --from=espidf-builder /root/.espressif /root/.espressif
|
|
|
|
|
|
2026-04-02 08:41:52 +07:00
|
|
|
COPY --from=espidf-builder /opt/arduino-esp32 /opt/arduino-esp32
|
|
|
|
|
|
2026-04-01 06:53:56 +07:00
|
|
|
ENV IDF_PATH=/opt/esp-idf
|
|
|
|
|
ENV IDF_TOOLS_PATH=/root/.espressif
|
2026-04-02 08:41:52 +07:00
|
|
|
ENV ARDUINO_ESP32_PATH=/opt/arduino-esp32
|
2026-04-01 06:53:56 +07:00
|
|
|
|
perf(espidf): drop in ccache for ESP32 compiles (~10× warm speedup)
Cold first compile per container is unchanged (cache empty). Subsequent
compiles drop from ~5-7 minutes to ~30-60 seconds because every ESP-IDF
base object (FreeRTOS, lwIP, esp_wifi, libsodium, soc, hal, …) hits the
cache. The user's BMP280 example, which hangs on cold compile, completes
near-instantly on the second attempt.
Why a transparent cache is safe: ccache hashes the preprocessed source +
flags + compiler. A cache hit only happens when the input is byte-for-byte
identical to a prior compile. Different sketches with different libraries
still get correct cache misses; there is no path where one project's
output contaminates another.
Changes
- Dockerfile.standalone: install ccache, set CCACHE_DIR=/var/cache/ccache,
IDF_CCACHE_ENABLE=1, configure 2 GB cap with compression. Compression
(level 6) cuts cache disk usage by ~40% with negligible CPU overhead.
- docker-compose.yml: named volume `ccache:/var/cache/ccache` so the
cache survives `docker compose up -d --build` (without it, every image
rebuild discards the cache).
- backend/app/services/espidf_compiler.py: pass `-DCCACHE_ENABLE=1` to
cmake when IDF_CCACHE_ENABLE is truthy. ESP-IDF's project.cmake
(`set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)` on line 374)
is what actually wires ccache in; without the cmake -D flag the env
var alone has no effect because we don't go through idf.py.
Escape hatch: set IDF_CCACHE_ENABLE=0 in compose env to disable without
rebuilding the image.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 10:44:29 +07:00
|
|
|
# ── ccache for ESP-IDF compiles ──────────────────────────────────────────────
|
|
|
|
|
# ESP-IDF's build system honours IDF_CCACHE_ENABLE=1 and routes every C/C++
|
|
|
|
|
# compile through ccache. Cold first compile per container is unchanged
|
|
|
|
|
# (cache is empty), but the second and subsequent compiles drop from
|
|
|
|
|
# ~5-7 minutes to ~30-60 seconds because every ESP-IDF base object
|
|
|
|
|
# (FreeRTOS, lwIP, esp_wifi, libsodium, …) hits the cache.
|
|
|
|
|
#
|
|
|
|
|
# Cache lives at /var/cache/ccache. Mount as a docker volume in
|
|
|
|
|
# docker-compose.yml so the cache survives `docker compose up -d --build`.
|
|
|
|
|
# Without the volume, the cache rebuilds itself on first compile after each
|
|
|
|
|
# image rebuild — still better than no cache.
|
|
|
|
|
ENV CCACHE_DIR=/var/cache/ccache
|
|
|
|
|
ENV IDF_CCACHE_ENABLE=1
|
|
|
|
|
RUN mkdir -p /var/cache/ccache \
|
|
|
|
|
&& ccache --max-size=2G \
|
|
|
|
|
&& ccache --set-config=compression=true \
|
|
|
|
|
&& ccache --set-config=compression_level=6
|
|
|
|
|
|
2026-04-01 12:45:47 +07:00
|
|
|
# Install ESP-IDF Python dependencies using the final image's Python
|
|
|
|
|
# The requirements.txt has version constraints required by ESP-IDF 4.4.x
|
|
|
|
|
RUN grep -v 'esp-windows-curses' /opt/esp-idf/requirements.txt \
|
|
|
|
|
| pip install --no-cache-dir -r /dev/stdin
|
|
|
|
|
|
2026-03-06 20:14:50 +07:00
|
|
|
EXPOSE 80
|
|
|
|
|
|
|
|
|
|
CMD ["/app/entrypoint.sh"]
|