LeRobot deployment adapter¶
LeRobot trains the policy, runs cameras, and talks to the robot. FlowEdge is the native action head behind that plugin: converted diffusion_pusht first, then a cached SmolVLA expert. The problem this solves is the last milliseconds — a fixed U-Net, a period, and an explicit miss policy — without putting e-stop or joint limits inside Core.
Boundary¶
LeRobot |
FlowEdge |
|---|---|
Cameras, RGB/state encoder, normalization, history |
Fixed-memory U-Net / expert |
Robot driver, joint limits, e-stop |
|
Training |
Converted |
Action slice: actions[observation_steps - 1 : observation_steps - 1 + action_steps].
input_mode="visual" uses the source encoder. A flattened condition remains valid. Policy evaluation.
Install¶
python -m pip install -e integrations/lerobot
python -m unittest discover -s integrations/lerobot/tests
from flowedge_lerobot import FlowEdgeDiffusionPolicy
policy = FlowEdgeDiffusionPolicy.from_checkpoint(
"models/diffusion_pusht.flowedge.safetensors", threads=4
)
Period loop (sim, Jetson, SO-100)¶
python -m flowedge_dev pipeline rollout models/diffusion_pusht.flowedge.safetensors \
--steps 20 --threads 4 --period-ms 10 --on-miss hold
CUDA Core (FLOWEDGE_BACKEND=cuda), same miss contract:
python -m flowedge_dev pipeline rollout models/diffusion_pusht.flowedge.safetensors \
--steps 20 --threads 1 --period-ms 10 --on-miss hold --device cuda
--device cuda requires a CUDA Core binary and Engine.cuda_resident. A 4 GB
card that cannot upload the U-Net stays on CPU kernels; that is a load success,
not a CUDA replay. FLOWEDGE_CUDA_REQUIRED=1 fails closed. GTX 1650, 10 ms hold:
20 / 20 misses. Not a Jetson/ARM claim.
|
On overrun |
|---|---|
|
Repeat last sent action; zeros before the first on-time send |
|
Skip |
|
|
Jetson: scripts/edge_dp_rollout.sh. Attach JSON to issue #70. SO-100: implement reset / observe / send_action / stop and pass encode_condition into run_rollout. CI uses a fake robot. ARM64 CI is architecture validation, not a board claim.
--policy.type=flowedge is DP. --policy.type=flowedge_smolvla is the native expert plus a LeRobot VLM cache — not a native VLM. Tenstorrent is a separate backend, not this plugin.