feat(simulator): tilt and battery controls for boards that carry an IMU or a gauge
The emulated BMI270 and battery gauge were faithful and inert: with nothing driving them the IMU reported the board lying flat for the whole run and the gauge reported the single voltage it was seeded with, so a sketch that reacts to tilt or warns on low charge could not be exercised at all. A Sensors popover on the canvas header supplies the missing half. Tilt is entered as roll/pitch on a drag pad rather than three raw axes — what a user means is "lean it left" — and gravity in the board's frame is that rotation applied to (0, 0, 1) g, which is also why the vector stays at 1 g whatever the attitude. Turning the board spins the gyro for as long as the drag lasts and parks it back at zero shortly after, matching a real part: a held tilt reads no rotation. Battery is a slider over the LiPo working range. Gated per board by builtInImu / builtInBattery on the pro board def, the same way the Mic toggle is gated, and each section renders only if its flag is set. The microphone stream also learns to resample. The engine now tells the source which rate the guest programmed, so a sketch recording at 8 kHz consumes two captured samples per read instead of playing back at half speed.
This commit is contained in:
David Montero Crespo
parent
d446031674
commit
7a7c9f93f7
|
|
@ -0,0 +1,49 @@
|
|||
/**
|
||||
* The tilt pad's gravity maths, on its own.
|
||||
*
|
||||
* What a user aims with the pad is an attitude; what the IMU model wants is
|
||||
* the gravity vector in the board's frame. Getting the mapping wrong is the
|
||||
* kind of bug that looks fine on screen — the dot moves, numbers change — and
|
||||
* only shows up as a sketch tilting the wrong way, so it is pinned here rather
|
||||
* than left to the eye.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { gravityFor } from '../components/simulator/BoardSensorControls';
|
||||
|
||||
/** Length of the vector: gravity is 1 g however the board is held. */
|
||||
const magnitude = ([x, y, z]: [number, number, number]) => Math.sqrt(x * x + y * y + z * z);
|
||||
|
||||
describe('tilt pad gravity', () => {
|
||||
it('reads flat on a table as +1 g on Z', () => {
|
||||
const [x, y, z] = gravityFor(0, 0);
|
||||
expect(x).toBeCloseTo(0, 6);
|
||||
expect(y).toBeCloseTo(0, 6);
|
||||
expect(z).toBeCloseTo(1, 6);
|
||||
});
|
||||
|
||||
it('stays at 1 g whatever the attitude', () => {
|
||||
for (const roll of [-80, -37, 0, 12, 80]) {
|
||||
for (const pitch of [-80, -5, 0, 45, 80]) {
|
||||
expect(magnitude(gravityFor(roll, pitch))).toBeCloseTo(1, 6);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('rolls onto -X and pitches onto +Y', () => {
|
||||
const rolled = gravityFor(90, 0);
|
||||
expect(rolled[0]).toBeCloseTo(-1, 6);
|
||||
expect(rolled[2]).toBeCloseTo(0, 6);
|
||||
|
||||
const pitched = gravityFor(0, 90);
|
||||
expect(pitched[1]).toBeCloseTo(1, 6);
|
||||
expect(pitched[2]).toBeCloseTo(0, 6);
|
||||
});
|
||||
|
||||
it('mirrors opposite tilts', () => {
|
||||
const [lx, ly, lz] = gravityFor(-30, -20);
|
||||
const [rx, ry, rz] = gravityFor(30, 20);
|
||||
expect(lx).toBeCloseTo(-rx, 6);
|
||||
expect(ly).toBeCloseTo(-ry, 6);
|
||||
expect(lz).toBeCloseTo(rz, 6); // Z is even in both angles
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,315 @@
|
|||
/**
|
||||
* Canvas-header popover for the simulation-only inputs of a board's on-board
|
||||
* sensors: which way the device is being held, and how full its battery is.
|
||||
*
|
||||
* These exist because the emulated parts are perfectly faithful and therefore
|
||||
* perfectly boring on their own — an IMU with nothing driving it reports the
|
||||
* board lying flat forever, and a battery gauge reports the one voltage it was
|
||||
* seeded with. A sketch that reacts to tilt or warns on low charge then has no
|
||||
* way to be exercised. The controls are the missing half of those models.
|
||||
*
|
||||
* Gated per board by the pro board def (builtInImu / builtInBattery), the same
|
||||
* way the Mic toggle is, and each section renders only if its flag is set.
|
||||
* Values go straight to the board's bridge (setImuAcceleration / setImuGyro /
|
||||
* setBatteryVoltage); a board whose bridge lacks a seam simply ignores it.
|
||||
*
|
||||
* Tilt is entered as roll/pitch on a drag pad rather than as three raw axes:
|
||||
* what a user wants to say is "lean it left", and gravity in the board's own
|
||||
* frame is then just its rotation applied to (0, 0, 1) g. Rotating the board
|
||||
* also spins the gyro for as long as the drag lasts, which is what a real
|
||||
* movement produces — a held tilt reads zero rotation.
|
||||
*/
|
||||
import React, { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { getEsp32Bridge } from '../../store/useSimulatorStore';
|
||||
|
||||
interface SensorBridge {
|
||||
setImuAcceleration?: (x: number, y: number, z: number) => void;
|
||||
setImuGyro?: (x: number, y: number, z: number) => void;
|
||||
setBatteryVoltage?: (millivolts: number) => void;
|
||||
}
|
||||
|
||||
interface BoardSensorControlsProps {
|
||||
boardId: string | null;
|
||||
/** Board carries an IMU whose bridge accepts injected motion. */
|
||||
showImu: boolean;
|
||||
/** Board reads its pack voltage through the bridge. */
|
||||
showBattery: boolean;
|
||||
}
|
||||
|
||||
/** Degrees of tilt at the edge of the pad. Past ~80 degrees gravity is nearly
|
||||
* in the plane of the board and the pad stops being a useful way to aim it. */
|
||||
const MAX_TILT_DEG = 80;
|
||||
const PAD_PX = 132;
|
||||
|
||||
/** LiPo working range: 3.3 V is where M5Unified's UI starts calling it empty,
|
||||
* 4.2 V is a full pack straight off the charger. */
|
||||
const BATT_MIN_MV = 3300;
|
||||
const BATT_MAX_MV = 4200;
|
||||
|
||||
const RAD = Math.PI / 180;
|
||||
|
||||
/**
|
||||
* Gravity in the board's frame for a given roll (Y tilt) and pitch (X tilt).
|
||||
* Flat is (0, 0, 1) g; rolling right pushes gravity onto -X, pitching forward
|
||||
* onto +Y — the sign convention M5Unified reports for this part, so a sketch
|
||||
* written against the real board reads the same numbers here.
|
||||
*/
|
||||
export function gravityFor(rollDeg: number, pitchDeg: number): [number, number, number] {
|
||||
const r = rollDeg * RAD;
|
||||
const p = pitchDeg * RAD;
|
||||
return [-Math.sin(r) * Math.cos(p), Math.sin(p), Math.cos(r) * Math.cos(p)];
|
||||
}
|
||||
|
||||
export const BoardSensorControls: React.FC<BoardSensorControlsProps> = ({
|
||||
boardId,
|
||||
showImu,
|
||||
showBattery,
|
||||
}) => {
|
||||
const [open, setOpen] = useState(false);
|
||||
const [roll, setRoll] = useState(0);
|
||||
const [pitch, setPitch] = useState(0);
|
||||
const [batteryMv, setBatteryMv] = useState(4000);
|
||||
const padRef = useRef<HTMLDivElement | null>(null);
|
||||
const draggingRef = useRef(false);
|
||||
/** Previous angles + timestamp, so a drag can be turned into a rotation rate. */
|
||||
const lastRef = useRef<{ roll: number; pitch: number; t: number } | null>(null);
|
||||
const gyroStopRef = useRef<number | null>(null);
|
||||
|
||||
const bridge = useCallback(
|
||||
() => (boardId ? (getEsp32Bridge(boardId) as SensorBridge | undefined) : undefined),
|
||||
[boardId],
|
||||
);
|
||||
|
||||
// Push the current attitude whenever it changes — including on mount and on
|
||||
// every re-run, since a fresh bridge starts back at flat-on-table.
|
||||
useEffect(() => {
|
||||
if (!showImu) return;
|
||||
const [x, y, z] = gravityFor(roll, pitch);
|
||||
bridge()?.setImuAcceleration?.(x, y, z);
|
||||
}, [roll, pitch, showImu, bridge]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!showBattery) return;
|
||||
bridge()?.setBatteryVoltage?.(batteryMv);
|
||||
}, [batteryMv, showBattery, bridge]);
|
||||
|
||||
// A rotation rate only exists while the board is actually being turned. The
|
||||
// timer parks the gyro back at zero shortly after the drag stops, which is
|
||||
// what a real part reports once the movement ends.
|
||||
const spinGyro = useCallback(
|
||||
(nextRoll: number, nextPitch: number) => {
|
||||
const now = performance.now();
|
||||
const last = lastRef.current;
|
||||
lastRef.current = { roll: nextRoll, pitch: nextPitch, t: now };
|
||||
if (!last) return;
|
||||
const dt = Math.max(1, now - last.t) / 1000;
|
||||
const gx = (nextPitch - last.pitch) / dt;
|
||||
const gy = (nextRoll - last.roll) / dt;
|
||||
bridge()?.setImuGyro?.(gx, gy, 0);
|
||||
if (gyroStopRef.current !== null) window.clearTimeout(gyroStopRef.current);
|
||||
gyroStopRef.current = window.setTimeout(() => {
|
||||
bridge()?.setImuGyro?.(0, 0, 0);
|
||||
lastRef.current = null;
|
||||
}, 150);
|
||||
},
|
||||
[bridge],
|
||||
);
|
||||
|
||||
useEffect(
|
||||
() => () => {
|
||||
if (gyroStopRef.current !== null) window.clearTimeout(gyroStopRef.current);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
const applyPointer = useCallback(
|
||||
(clientX: number, clientY: number) => {
|
||||
const pad = padRef.current;
|
||||
if (!pad) return;
|
||||
const r = pad.getBoundingClientRect();
|
||||
// Clamp to the pad: a drag that leaves the box keeps tilting to the edge
|
||||
// rather than snapping back to centre.
|
||||
const nx = Math.max(-1, Math.min(1, ((clientX - r.left) / r.width) * 2 - 1));
|
||||
const ny = Math.max(-1, Math.min(1, ((clientY - r.top) / r.height) * 2 - 1));
|
||||
const nextRoll = Math.round(nx * MAX_TILT_DEG);
|
||||
const nextPitch = Math.round(ny * MAX_TILT_DEG);
|
||||
setRoll(nextRoll);
|
||||
setPitch(nextPitch);
|
||||
spinGyro(nextRoll, nextPitch);
|
||||
},
|
||||
[spinGyro],
|
||||
);
|
||||
|
||||
const onPointerDown = (e: React.PointerEvent) => {
|
||||
draggingRef.current = true;
|
||||
(e.target as HTMLElement).setPointerCapture?.(e.pointerId);
|
||||
applyPointer(e.clientX, e.clientY);
|
||||
};
|
||||
const onPointerMove = (e: React.PointerEvent) => {
|
||||
if (draggingRef.current) applyPointer(e.clientX, e.clientY);
|
||||
};
|
||||
const onPointerUp = () => {
|
||||
draggingRef.current = false;
|
||||
};
|
||||
|
||||
const resetTilt = () => {
|
||||
setRoll(0);
|
||||
setPitch(0);
|
||||
lastRef.current = null;
|
||||
bridge()?.setImuGyro?.(0, 0, 0);
|
||||
};
|
||||
|
||||
const [ax, ay, az] = gravityFor(roll, pitch);
|
||||
const dotX = (roll / MAX_TILT_DEG) * (PAD_PX / 2 - 10);
|
||||
const dotY = (pitch / MAX_TILT_DEG) * (PAD_PX / 2 - 10);
|
||||
const battPct = Math.round(((batteryMv - BATT_MIN_MV) / (BATT_MAX_MV - BATT_MIN_MV)) * 100);
|
||||
|
||||
return (
|
||||
<div style={{ position: 'relative' }}>
|
||||
<button
|
||||
onClick={() => setOpen((v) => !v)}
|
||||
disabled={!boardId}
|
||||
title="Tilt the board and set its battery level (simulation inputs)"
|
||||
style={{
|
||||
backgroundColor: open ? 'rgba(88,166,255,0.15)' : 'transparent',
|
||||
border: `1px solid ${open ? '#58a6ff' : 'transparent'}`,
|
||||
borderRadius: 4,
|
||||
padding: '4px 10px',
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
gap: 6,
|
||||
color: open ? '#58a6ff' : '#ccc',
|
||||
fontSize: 13,
|
||||
cursor: boardId ? 'pointer' : 'not-allowed',
|
||||
}}
|
||||
>
|
||||
<svg
|
||||
width="16"
|
||||
height="16"
|
||||
viewBox="0 0 24 24"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
>
|
||||
<circle cx="12" cy="12" r="9" />
|
||||
<path d="M15.5 8.5l-2 5-5 2 2-5z" />
|
||||
</svg>
|
||||
<span>Sensors</span>
|
||||
</button>
|
||||
|
||||
{open && (
|
||||
<div
|
||||
style={{
|
||||
position: 'absolute',
|
||||
top: 'calc(100% + 6px)',
|
||||
right: 0,
|
||||
zIndex: 40,
|
||||
background: '#161b22',
|
||||
border: '1px solid #30363d',
|
||||
borderRadius: 6,
|
||||
padding: 12,
|
||||
width: 200,
|
||||
boxShadow: '0 6px 20px rgba(0,0,0,0.45)',
|
||||
color: '#c9d1d9',
|
||||
fontSize: 12,
|
||||
}}
|
||||
>
|
||||
{showImu && (
|
||||
<>
|
||||
<div
|
||||
style={{
|
||||
display: 'flex',
|
||||
justifyContent: 'space-between',
|
||||
alignItems: 'baseline',
|
||||
marginBottom: 6,
|
||||
}}
|
||||
>
|
||||
<strong style={{ fontSize: 12 }}>Tilt</strong>
|
||||
<button
|
||||
onClick={resetTilt}
|
||||
style={{
|
||||
background: 'none',
|
||||
border: 'none',
|
||||
color: '#58a6ff',
|
||||
cursor: 'pointer',
|
||||
fontSize: 11,
|
||||
padding: 0,
|
||||
}}
|
||||
>
|
||||
flat
|
||||
</button>
|
||||
</div>
|
||||
<div
|
||||
ref={padRef}
|
||||
onPointerDown={onPointerDown}
|
||||
onPointerMove={onPointerMove}
|
||||
onPointerUp={onPointerUp}
|
||||
onPointerCancel={onPointerUp}
|
||||
style={{
|
||||
position: 'relative',
|
||||
width: PAD_PX,
|
||||
height: PAD_PX,
|
||||
margin: '0 auto',
|
||||
borderRadius: '50%',
|
||||
border: '1px solid #30363d',
|
||||
background:
|
||||
'radial-gradient(circle at 50% 50%, #21262d 0%, #161b22 70%, #0d1117 100%)',
|
||||
cursor: 'grab',
|
||||
touchAction: 'none',
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
position: 'absolute',
|
||||
left: '50%',
|
||||
top: '50%',
|
||||
width: 14,
|
||||
height: 14,
|
||||
marginLeft: -7 + dotX,
|
||||
marginTop: -7 + dotY,
|
||||
borderRadius: '50%',
|
||||
background: '#58a6ff',
|
||||
boxShadow: '0 0 8px rgba(88,166,255,0.6)',
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
<div
|
||||
style={{
|
||||
marginTop: 6,
|
||||
fontFamily: 'monospace',
|
||||
fontSize: 11,
|
||||
color: '#8b949e',
|
||||
textAlign: 'center',
|
||||
}}
|
||||
>
|
||||
{ax.toFixed(2)} / {ay.toFixed(2)} / {az.toFixed(2)} g
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
{showBattery && (
|
||||
<div style={{ marginTop: showImu ? 12 : 0 }}>
|
||||
<div style={{ display: 'flex', justifyContent: 'space-between', marginBottom: 4 }}>
|
||||
<strong style={{ fontSize: 12 }}>Battery</strong>
|
||||
<span style={{ fontFamily: 'monospace', color: '#8b949e' }}>
|
||||
{batteryMv} mV - {battPct}%
|
||||
</span>
|
||||
</div>
|
||||
<input
|
||||
type="range"
|
||||
min={BATT_MIN_MV}
|
||||
max={BATT_MAX_MV}
|
||||
step={10}
|
||||
value={batteryMv}
|
||||
onChange={(e) => setBatteryMv(Number(e.target.value))}
|
||||
style={{ width: '100%' }}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
|
@ -23,6 +23,7 @@ import { getTabSessionId } from '../../simulation/Esp32Bridge';
|
|||
import { CameraToggle } from './CameraToggle';
|
||||
import { ComponentCameraToggles } from './ComponentCameraToggles';
|
||||
import { MicrophoneToggle } from './MicrophoneToggle';
|
||||
import { BoardSensorControls } from './BoardSensorControls';
|
||||
import { WireLayer } from './WireLayer';
|
||||
import type { SegmentHandle, WaypointHandle, AlignmentGuide } from './WireLayer';
|
||||
import { ElectricalOverlay } from '../analog-ui/ElectricalOverlay';
|
||||
|
|
@ -2749,6 +2750,24 @@ export const SimulatorCanvas = ({ headerSlot }: SimulatorCanvasProps = {}) => {
|
|||
<MicrophoneToggle boardId={activeBoard.id} />
|
||||
)}
|
||||
|
||||
{/* Tilt pad + battery slider, for boards whose def declares an
|
||||
IMU or a battery gauge. Purely simulation inputs: the
|
||||
emulated parts are faithful but static without them (an
|
||||
untouched IMU reports the board flat on a table forever). */}
|
||||
{activeBoard &&
|
||||
(() => {
|
||||
const def = getProBoard(activeBoard.boardKind);
|
||||
const imu = def?.builtInImu === true;
|
||||
const battery = def?.builtInBattery === true;
|
||||
return imu || battery ? (
|
||||
<BoardSensorControls
|
||||
boardId={activeBoard.id}
|
||||
showImu={imu}
|
||||
showBattery={battery}
|
||||
/>
|
||||
) : null;
|
||||
})()}
|
||||
|
||||
{/* WiFi status indicator + IoT-gateway launcher (ESP32 + Pico W) */}
|
||||
{activeBoard &&
|
||||
(isEsp32Kind(activeBoard.boardKind) || activeBoard.boardKind === 'pi-pico-w') &&
|
||||
|
|
|
|||
|
|
@ -8,14 +8,15 @@
|
|||
* - status: 'idle' | 'requesting' | 'streaming' | 'denied' | 'error'.
|
||||
*
|
||||
* The transport is the bridge's setMicrophoneSource(fn) seam: the engine's
|
||||
* I2S RX fill pulls one signed 16-bit sample per call at 16 kHz. Boards whose
|
||||
* bridge lacks the seam (QEMU path, non-audio bridges) are reported as an
|
||||
* error instead of pretending to stream.
|
||||
* I2S RX fill pulls one signed 16-bit sample per call, passing the rate the
|
||||
* guest programmed. Boards whose bridge lacks the seam (QEMU path, non-audio
|
||||
* bridges) are reported as an error instead of pretending to stream.
|
||||
*
|
||||
* Implementation notes:
|
||||
* - Capture runs at the AudioContext's native rate (typically 48 kHz) and
|
||||
* is decimated to 16 kHz by fractional stepping — the emulated PDM mic is
|
||||
* a level-meter-grade source, not a hi-fi path.
|
||||
* a level-meter-grade source, not a hi-fi path. The reader then resamples
|
||||
* that ring to whatever rate the sketch asked for.
|
||||
* - The engine paces its pulls in SIMULATED cpu cycles, so during the boot
|
||||
* boost it can consume up to 4x faster than real time. A 2 s ring buffer
|
||||
* absorbs jitter; underflow reads as silence (0), overflow drops the
|
||||
|
|
@ -44,7 +45,9 @@ const MIC_HZ = 16_000;
|
|||
const RING_CAPACITY = MIC_HZ * 2;
|
||||
|
||||
interface MicBridge {
|
||||
setMicrophoneSource?: (s: (() => number) | null) => void;
|
||||
/** The source is called once per sample and is handed the rate the guest
|
||||
* configured, so it can resample the captured audio to match. */
|
||||
setMicrophoneSource?: (s: ((hz?: number) => number) | null) => void;
|
||||
}
|
||||
|
||||
export function useMicrophoneStream(): UseMicrophoneStreamResult {
|
||||
|
|
@ -156,11 +159,22 @@ export function useMicrophoneStream(): UseMicrophoneStreamResult {
|
|||
// output buffer stays zeroed, so nothing is audible.
|
||||
processor.connect(ctx.destination);
|
||||
|
||||
const sourceFn = () => {
|
||||
// How much of the CAPTURED stream one guest sample covers. The ring is
|
||||
// written at MIC_HZ, so a sketch recording at 8 kHz must consume two
|
||||
// captured samples per read or its audio plays back at half speed, and
|
||||
// one at 44.1 kHz consumes less than one (zero-order hold). The engine
|
||||
// passes the rate it decoded from the guest's own clock registers.
|
||||
let readFrac = 0;
|
||||
const sourceFn = (hz?: number) => {
|
||||
if (count === 0) return 0; // starved: emulation pulling ahead of real time
|
||||
const s = ring[readPos];
|
||||
readPos = (readPos + 1) % RING_CAPACITY;
|
||||
count--;
|
||||
readFrac += hz && hz > 0 ? MIC_HZ / hz : 1;
|
||||
const advance = Math.floor(readFrac);
|
||||
readFrac -= advance;
|
||||
for (let k = 0; k < advance && count > 0; k++) {
|
||||
readPos = (readPos + 1) % RING_CAPACITY;
|
||||
count--;
|
||||
}
|
||||
return s;
|
||||
};
|
||||
bridge.setMicrophoneSource(sourceFn);
|
||||
|
|
|
|||
|
|
@ -105,6 +105,13 @@ export interface ProBoardDef {
|
|||
* setMicrophoneSource (I2S RX sample injection): shows the canvas-header
|
||||
* Mic toggle that streams the computer's microphone into it. */
|
||||
builtInMicrophone?: boolean;
|
||||
/** Board carries an IMU whose bridge implements setImuAcceleration /
|
||||
* setImuGyro: shows the canvas-header tilt pad. Without it the emulated
|
||||
* part faithfully reports the board lying flat for the whole run. */
|
||||
builtInImu?: boolean;
|
||||
/** Board reads a battery through its bridge (setBatteryVoltage): shows the
|
||||
* canvas-header charge slider, so low-battery code paths can be reached. */
|
||||
builtInBattery?: boolean;
|
||||
/** Built-in peripheral attachment for a RUNNING board (LCD decoder onto the
|
||||
* element's own canvas, speaker, on-board button/keyboard event forwarding).
|
||||
* Called shortly after run start with the board's DOM element plus its
|
||||
|
|
|
|||
Loading…
Reference in New Issue