Architecture

Runtime boundaries

Python or another controller
          │
          │ WebSocket / TCP / UDP
          ▼
Local Sim2Bot bridge
          │
          │ WebSocket control + telemetry
          │ separate binary camera stream
          ▼
Browser app: React UI + Three.js + MuJoCo WASM

The browser owns scene state, MuJoCo simulation, rendering, camera production, and GUI telemetry. The bridge adapts browser-compatible WebSockets to controller interfaces. The Python SDK provides typed, synchronous control and latest-value state/camera abstractions.

Important ownership rules

• Robot commands and robot telemetry use a robot index discovered from the scene.
• Cameras are global scene resources and expose mount metadata independently.
• The bridge relays state; it is not the physics engine or authoritative scene store.
• Cloud account data and local scene assets have separate persistence concerns.

Timing model

MuJoCo timestep controls simulated physics resolution. The browser frame loop currently schedules batches of physics steps and applies newly received commands. Telemetry can carry substep samples, but the surrounding browser runtime is not a deterministic wall-clock servo.

The planned worker-based runtime is documented in the repository's docs/TODO/worker-physics-runtime.md; it is deferred until measurements justify the additional concurrency and synchronization complexity.