velxio/frontend/src/utils/sdCardFiles.ts

87 lines
2.9 KiB
TypeScript
Raw Normal View History

feat(microsd): SD-over-SPI card storage for AVR, RP2040 and ESP32 Add a working microSD card part backed by a FAT16 image, following the Wokwi storage model: the project's own workspace files are auto-copied onto the card (free), and an optional "SD Card" panel uploads extra files (gated as a paid feature by the velxio.dev overlay; OSS default allows it). Frontend (in-browser AVR / RP2040): - ProtocolParts.ts: rewrite the microsd-card part from a handshake stub into a real SD-over-SPI device (reply-first Ncr timing, SDSC byte addressing, single/multi-block read+write, CSD/CID, full CMD set). - utils/fatImage.ts: dependency-free FAT16 super-floppy builder (8.3 + LFN). - utils/sdCardFiles.ts: assemble the card image from workspace files plus uploaded files; base64 helpers. - components/simulator/SdCardPanel.tsx + ComponentPropertyDialog: upload UI. - DynamicComponent + useSimulatorStore: build and inject the image on run. - lib/proSdCardGate.ts: overlay-installable gate for the upload action. - data/examples-storage-microsd.ts: Arduino Uno + ESP32 gallery examples. Backend (ESP32 via QEMU): - services/esp32_sd_slave.py: synchronous SD-over-SPI slave (Python port of the browser part) with a sparse backing store, idle-state R1 tracking and real CRC16 on data blocks when the host enables CRC (CMD59) -- both required by ESP-IDF's sdspi driver. - esp32_worker.py: route SPI bytes to the slave (returns MISO synchronously) and feed write-only bulk transfers. - esp32_lib_manager.py + routes/simulation.py: forward the FAT image (sd_card.image_b64) from the start config into the worker. Tested: - frontend: protocol-parts, fat-image, sd-card-gate and microsd-real-firmware (real Arduino SD.h on avr8js) -- 86 passing. - backend: test_esp32_sd_slave (10) covering the ESP-IDF init sequence and CRC16; validated end to end by running a real SD.h sketch in libqemu-xtensa (mount, directory listing, read and write-readback).
2026-06-11 08:59:53 +07:00
/**
* sdCardFiles.ts — assemble the microSD card's FAT16 image from the files that
* should live on it.
*
* Two sources, matching the Wokwi model:
* - FREE: the project's workspace files are auto-copied (text sources, etc.).
* - PAID: user-uploaded files (binaries included) from the "SD Card" panel.
*
* Uploaded files win over a project file of the same name. The result is a
* Uint8Array disk image handed to the `microsd-card` simulation part via
* `element.sdImageData`.
*/
import { buildFat16Image, type SdFile, type BuildFatOptions } from './fatImage';
export interface WorkspaceFileLike {
name: string;
content: string;
}
/** An uploaded SD file as persisted on the component (`properties.sdFiles`). */
export interface UploadedSdFile {
name: string;
/** Base64 of the raw bytes (binaries included). */
contentB64: string;
}
/** Max total size of uploaded files — matches the FAT16 volume (Wokwi ~8 MB). */
export const SD_UPLOAD_MAX_BYTES = 8 * 1024 * 1024;
function b64ToBytes(b64: string): Uint8Array {
const bin = atob(b64);
const out = new Uint8Array(bin.length);
for (let i = 0; i < bin.length; i++) out[i] = bin.charCodeAt(i);
return out;
}
export function bytesToB64(bytes: Uint8Array): string {
let bin = '';
const CHUNK = 0x8000;
for (let i = 0; i < bytes.length; i += CHUNK) {
bin += String.fromCharCode(...bytes.subarray(i, i + CHUNK));
}
return btoa(bin);
}
/** Decode the persisted `properties.sdFiles` into builder-ready SdFile[]. */
export function decodeSdFiles(raw: unknown): SdFile[] {
if (!Array.isArray(raw)) return [];
const out: SdFile[] = [];
for (const e of raw) {
if (e && typeof e.name === 'string' && typeof e.contentB64 === 'string') {
try {
out.push({ name: e.name, data: b64ToBytes(e.contentB64) });
} catch {
/* skip malformed entry */
}
}
}
return out;
}
/** Source files never land on the card: on real hardware your code lives in
* flash, not on the microSD, and a sketch listing the card should see its
* DATA (csv, json, txt, wav...), not its own source. Uploads are exempt —
* an explicit upload always lands, whatever its extension. */
const SD_EXCLUDED_SOURCES = /\.(ino|h|hpp|c|cpp|py)$/i;
feat(microsd): SD-over-SPI card storage for AVR, RP2040 and ESP32 Add a working microSD card part backed by a FAT16 image, following the Wokwi storage model: the project's own workspace files are auto-copied onto the card (free), and an optional "SD Card" panel uploads extra files (gated as a paid feature by the velxio.dev overlay; OSS default allows it). Frontend (in-browser AVR / RP2040): - ProtocolParts.ts: rewrite the microsd-card part from a handshake stub into a real SD-over-SPI device (reply-first Ncr timing, SDSC byte addressing, single/multi-block read+write, CSD/CID, full CMD set). - utils/fatImage.ts: dependency-free FAT16 super-floppy builder (8.3 + LFN). - utils/sdCardFiles.ts: assemble the card image from workspace files plus uploaded files; base64 helpers. - components/simulator/SdCardPanel.tsx + ComponentPropertyDialog: upload UI. - DynamicComponent + useSimulatorStore: build and inject the image on run. - lib/proSdCardGate.ts: overlay-installable gate for the upload action. - data/examples-storage-microsd.ts: Arduino Uno + ESP32 gallery examples. Backend (ESP32 via QEMU): - services/esp32_sd_slave.py: synchronous SD-over-SPI slave (Python port of the browser part) with a sparse backing store, idle-state R1 tracking and real CRC16 on data blocks when the host enables CRC (CMD59) -- both required by ESP-IDF's sdspi driver. - esp32_worker.py: route SPI bytes to the slave (returns MISO synchronously) and feed write-only bulk transfers. - esp32_lib_manager.py + routes/simulation.py: forward the FAT image (sd_card.image_b64) from the start config into the worker. Tested: - frontend: protocol-parts, fat-image, sd-card-gate and microsd-real-firmware (real Arduino SD.h on avr8js) -- 86 passing. - backend: test_esp32_sd_slave (10) covering the ESP-IDF init sequence and CRC16; validated end to end by running a real SD.h sketch in libqemu-xtensa (mount, directory listing, read and write-readback).
2026-06-11 08:59:53 +07:00
/**
* Build the SD image from the project workspace files (auto-copied, free) plus
* any uploaded files (paid). Uploaded files override same-named project files.
*/
export function buildProjectSdImage(
workspaceFiles: WorkspaceFileLike[],
uploaded: SdFile[] = [],
opts?: BuildFatOptions,
): Uint8Array {
const byName = new Map<string, SdFile>();
for (const f of workspaceFiles) {
if (SD_EXCLUDED_SOURCES.test(f.name)) continue;
feat(microsd): SD-over-SPI card storage for AVR, RP2040 and ESP32 Add a working microSD card part backed by a FAT16 image, following the Wokwi storage model: the project's own workspace files are auto-copied onto the card (free), and an optional "SD Card" panel uploads extra files (gated as a paid feature by the velxio.dev overlay; OSS default allows it). Frontend (in-browser AVR / RP2040): - ProtocolParts.ts: rewrite the microsd-card part from a handshake stub into a real SD-over-SPI device (reply-first Ncr timing, SDSC byte addressing, single/multi-block read+write, CSD/CID, full CMD set). - utils/fatImage.ts: dependency-free FAT16 super-floppy builder (8.3 + LFN). - utils/sdCardFiles.ts: assemble the card image from workspace files plus uploaded files; base64 helpers. - components/simulator/SdCardPanel.tsx + ComponentPropertyDialog: upload UI. - DynamicComponent + useSimulatorStore: build and inject the image on run. - lib/proSdCardGate.ts: overlay-installable gate for the upload action. - data/examples-storage-microsd.ts: Arduino Uno + ESP32 gallery examples. Backend (ESP32 via QEMU): - services/esp32_sd_slave.py: synchronous SD-over-SPI slave (Python port of the browser part) with a sparse backing store, idle-state R1 tracking and real CRC16 on data blocks when the host enables CRC (CMD59) -- both required by ESP-IDF's sdspi driver. - esp32_worker.py: route SPI bytes to the slave (returns MISO synchronously) and feed write-only bulk transfers. - esp32_lib_manager.py + routes/simulation.py: forward the FAT image (sd_card.image_b64) from the start config into the worker. Tested: - frontend: protocol-parts, fat-image, sd-card-gate and microsd-real-firmware (real Arduino SD.h on avr8js) -- 86 passing. - backend: test_esp32_sd_slave (10) covering the ESP-IDF init sequence and CRC16; validated end to end by running a real SD.h sketch in libqemu-xtensa (mount, directory listing, read and write-readback).
2026-06-11 08:59:53 +07:00
byName.set(f.name.toLowerCase(), { name: f.name, data: new TextEncoder().encode(f.content) });
}
for (const f of uploaded) {
byName.set(f.name.toLowerCase(), f);
}
return buildFat16Image([...byName.values()], opts);
}