Command-line interface

The inspect_robots CLI wraps the registry and eval. The command is installed as inspect-robots, with inspect-robot as an alias for the common typo; both run the same CLI.

Zero-config: inspect-robots "<instruction>"

Once you have configured a default policy and embodiment (run inspect-robots setup, or see below), giving the robot a command is a one-liner:

inspect-robots "place the spoon on the plate"

This runs a single ad-hoc scene built from that language instruction on your default policy/embodiment: sugar for inspect-robots run --instruction "...". The resolved components and where they came from are printed before the robot moves. Two flags exist only for instruction runs: --max-steps N (horizon, default 300) and --scorer NAME (default operator).

The sugar only fires when the first argument contains whitespace, so a mistyped subcommand (inspect-robots isnpect) errors out instead of starting a rollout; a single-word instruction needs the explicit run --instruction "wipe" form.

Default policy and embodiment

Resolved in order (first hit wins):

1. explicit flags: --policy / --embodiment
2. environment: INSPECT_ROBOTS_POLICY / INSPECT_ROBOTS_EMBODIMENT
3. the user config file ~/.config/inspect-robots/config.ini ($XDG_CONFIG_HOME is honored):

[defaults]
policy = molmoact2
embodiment = yam_arms     ; the default: real hardware
sim_embodiment = my-sim   ; what --sim swaps in (any registered sim embodiment)
scorer = operator         ; optional, ad-hoc runs only
max_steps = 300           ; optional, ad-hoc runs only
store_frames = true       ; optional, capture frames on every run

[policy.args]          ; default -P key=value pairs
server_url = http://gpu-box:8202

[embodiment.args]      ; default -E key=value pairs
top_cam_device = /dev/v4l/by-id/usb-CAM123-video-index0

[sim_embodiment.args]  ; -E pairs used only under --sim
headless = true

Values parse like -P/-E args (bool/int/float/None/str), ~ expands in [*.args] values, and an explicit -P/-E key=value flag overrides the same-named config key. An [*.args] section belongs to the component named in [defaults]: it applies whenever that same component is the one selected (by default, by flag, or by env var), and is ignored with a stderr note when a different component is selected. Your YAM rig's rest_pose never reaches --embodiment kitchen. There is deliberately no project-local config file: a checked-in file choosing which policy drives your hardware would be a trust footgun.

Running in simulation: --sim

Real hardware is the default (it is whatever you configured as embodiment). --sim swaps in your configured sim counterpart for one invocation:

inspect-robots "place the spoon on the plate" --sim
inspect-robots run --task my-benchmark --policy molmoact2 --sim

The sim embodiment resolves as $INSPECT_ROBOTS_SIM_EMBODIMENT > config sim_embodiment, with constructor args from [sim_embodiment.args] only: real-rig args ([embodiment.args]: serial ports, camera IDs) never leak into a sim run, and vice versa. --sim together with an explicit --embodiment is an error (they both pick the embodiment); an exported $INSPECT_ROBOTS_EMBODIMENT is simply not consulted under --sim: it's a persistent default for real runs, not a per-invocation intent. The mapping is explicit configuration: the framework never guesses which sim matches your robot.

Operator scoring

An arbitrary instruction has no success oracle, so ad-hoc runs default to the operator scorer. When run on an interactive terminal, the CLI asks after each trial unless the embodiment already terminated the episode with a definitive success or failure verdict. In that case, the CLI records the embodiment's verdict as the operator judgement instead of asking the operator a second time, and prints operator verdict adopted from embodiment: success (or failure) so the operator can catch a mistaken adoption live.

did the robot succeed? [y/n/partial/skip] (partial scores as failure) n
grader notes (Enter for none): gripper closed early, cube still in frame

Prompted verdicts are recorded in the log. The CLI then asks for one optional line of grader notes. Bare Enter or whitespace-only input records no note. An adopted embodiment verdict is not followed by a notes prompt, so a self-scoring embodiment still costs no keypresses per trial. skip records no judgement, but a grader note entered for that trial is still recorded. Notes never affect the score. Piped/CI stdin or --no-prompt never prompt. A registered --task or eval-set run prompts only for trials that end with termination_reason="operator_end" — a human pressed the end-episode key — and never adopts an embodiment verdict or prompts for any other ending, so unattended runs stay non-blocking. An unjudged trial honestly scores as failure with "no operator judgement recorded".

inspect-robots setup

The interactive first-run wizard: it prompts for each [defaults] key with a suggested value (Enter accepts, typing overrides), warns when a chosen policy or embodiment is not registered in the current environment, and then helps assign camera devices. It lists every color-capable camera that udev names under /dev/v4l, preferring /dev/v4l/by-id names and falling back to port-stable /dev/v4l/by-path names when a by-id link is missing or when two cameras share one serial. Multi-interface cameras such as the RealSense D435 can lose udev's name race between their depth and RGB interfaces.

Answer u and unplug the camera when asked to identify the physical USB device that disappeared, including cameras the by-id listing cannot name. The wizard chooses the stored path after replug because udev reassigns links on every plug. Answer p to switch the listing to port names.

When the selected registered embodiment declares device slots, those slots drive one device interview for cameras, CAN interfaces, and serial devices. CAN slots list SocketCAN interfaces and support unplug-to-identify; rigs with multiple USB adapters named can0, can1, and so on also receive a udev pinning suggestion so replug order cannot swap physical devices.

inspect-robots setup

The result is written to ~/.config/inspect-robots/config.ini ($XDG_CONFIG_HOME honored); an existing file is backed up to config.ini.bak first, and settings the wizard does not manage (such as [policy.args] or sim_embodiment) are carried through unchanged. Note that later inspect-robots config set edits drop comments from the file. The setup command requires an interactive terminal; for scripted configuration use inspect-robots config set. After writing the config, setup lists missing runtime requirements declared by the selected registered policy and embodiment, together with their remediation commands.

Prefer to write the file yourself? This is the wizard's output for a YAM rig; replace the three camera paths with your rig's V4L2 color nodes (stable /dev/v4l/by-id/... or udev-symlink paths):

mkdir -p ~/.config/inspect-robots && cat > ~/.config/inspect-robots/config.ini <<'EOF'
[defaults]
policy = molmoact2        # from the inspect-robots-yam plugin
embodiment = yam_arms     # same plugin; cameras configured below
scorer = success_at_end
max_steps = 1200          # 120 s at 10 Hz
rerun = true              # live viewer of cameras/state/actions each run
store_frames = true       # save each run's camera frames under logs/frames/

[embodiment.args]
top_cam_device = /dev/v4l/by-id/YOUR-TOP-CAM
left_cam_device = /dev/v4l/by-id/YOUR-LEFT-CAM
right_cam_device = /dev/v4l/by-id/YOUR-RIGHT-CAM
EOF

inspect-robots list

Show registered components (builtins + installed plugins):

inspect-robots list                 # all kinds
inspect-robots list policies        # just one kind
inspect-robots list embodiments

inspect-robots run

Resolve a task/policy/embodiment from the registry and run an eval. Pass constructor arguments with -T (task), -P (policy), and -E (embodiment) as key=value (parsed as bool/int/float/None/str):

inspect-robots run --task cubepick-reach --policy scripted --embodiment cubepick
inspect-robots run --task cubepick-reach -T num_scenes=10 --policy scripted -P chunk_size=8 \
             --embodiment cubepick --log-dir logs --seed 0

--epochs N overrides the task's epoch count, --fail-on-error X halts on PolicyErrors (1 = first error, 0<X<1 = proportion, X>1 = count), and --store-frames streams camera frames to a per-run subdirectory of <log-dir>/frames (trial ids repeat across runs, so each run gets its own directory; the log's stats.frames_dir records the exact path). A store_frames = true config default enables capture on every run; --no-store-frames disables it for one invocation. When the run finishes, the path of the written log is printed.

--policy/--embodiment may be omitted when defaults are configured (see the zero-config section above); --instruction "..." replaces --task to run a single ad-hoc scene.

The exit code is 0 on a successful eval, 1 otherwise. When trials errored, the summary shows the count (trials: 4 (2 errored)) and lists each errored scene; a run in which every trial errored reports run status: error and exits 1.

inspect-robots eval-set

Run several registered tasks against one resolved policy/embodiment pair in a single invocation — the CLI counterpart of eval_set. Task names are matched exactly, or by shell-quoted fnmatch glob (entry-point discovery namespaces tasks as <benchmark>/<key>, so a benchmark name is a ready-made prefix):

inspect-robots eval-set 'kitchenbench/*' --policy xpolicylab -P url=ws://host:19000 \
             --embodiment yam_arms
inspect-robots eval-set cubepick-reach my-other-task --policy scripted --embodiment cubepick

For a task declared with max_seconds, its summary row includes both the physical-time budget and the integer step limit resolved from the selected embodiment, for example [120s -> 1200 steps at 10 Hz].

Multiple patterns may match the same task; it still runs once. A pattern that matches nothing is an error listing every registered task. --policy and --embodiment (and -P/-E, --sim, --epochs, --fail-on-error, --store-frames, --disable-guardrails, --max-action-delta) apply exactly as they do for run, to every matched task — there is no per-task -T in this release. The embodiment is resolved once for the whole set, not once per task, so a real robot is not reconnected between tasks.

Rather than one full summary per task, the CLI prints the resolved policy/embodiment, one status line for the whole set, a compact [status] task_name metrics-or-error row per task, and the shared log directory once (eval_set still writes one EvalLog per task inside it). The exit code is 0 iff every task's log has status == "success".

--retry-attempts is accepted and threaded through to eval_set(), whose resumption-of-a-partial-run behavior is reserved for a follow-up: passing it today does not yet skip already-finished scenes. --rerun's live viewer is not offered for eval-set: streaming several back-to-back tasks into one viewer window is a separate design question from running the set at all.

inspect-robots doctor

doctor reports a registered embodiment's missing declared runtime modules before constructing it, then checks its spaces for adapter conformance.

inspect-robots doctor --embodiment my_arms

inspect-robots inspect

Print a summary of a saved EvalLog:

inspect-robots inspect logs/cubepick-reach_xxxx.json

task:        cubepick-reach
policy:      scripted
embodiment:  cubepick
run status:  completed
outcome:     5 succeeded
horizon:     120s -> 1200 steps at 10 Hz
scenes:      5   trials: 5
metrics:
  success_at_end: 1
scenes:
  [success] scene-0: success_at_end=1
  ...

The horizon line appears for seconds-based tasks; step-only logs retain the existing output. The HTML viewer likewise separates declared seconds from the resolved step limit.

completed is the display form of the log's success status value; the on-disk field and Python API keep success.

For runs whose policy recorded conversations (such as --policy agent), --transcript appends each trial's recorded transcript after the summary:

inspect-robots inspect logs/cubepick-reach_xxxx.json --transcript

inspect-robots summarize

Distill a saved EvalLog into a markdown learnings file:

inspect-robots summarize logs/cubepick-reach_xxxx.json

Without --model, the command works offline and writes a deterministic digest of run identity, trial outcomes, operator feedback, errors, and transcript statistics. The default output is logs/learnings/cubepick-reach_xxxx.md. Use -o FILE to select another path or -o - to write only the document to stdout.

With --model, the digest and the tail of each recorded policy transcript are sent to an OpenAI-compatible chat-completions endpoint:

inspect-robots summarize logs/cubepick-reach_xxxx.json \
  --model claude-sonnet-4-5

The default endpoint is https://api.anthropic.com/v1, and the default API key variable is ANTHROPIC_API_KEY. Override them with --base-url URL and --api-key-env VAR for another compatible provider.

Retry with learning

The learnings file exists to be fed back in. The agent and capx policies accept a prior_learnings path and append the file's text to the system prompt after any embodiment notes, framed as hints that may be stale (the current observation always wins):

inspect-robots summarize logs/cubepick-reach_xxxx.json --model claude-sonnet-4-5
inspect-robots "place the fork on the plate" --policy agent \
    -P prior_learnings=logs/learnings/cubepick-reach_xxxx.md

The policy reads the file once when it is constructed and records its resolved path and content hash in the log's policy configuration, so runs that saw prior notes are never mistaken for cold-start runs when comparing results. Any hand-written markdown file works in place of a generated one. Validation details and limits live in the plugin READMEs linked above.

inspect-robots view

Render a saved EvalLog as a self-contained HTML report:

inspect-robots view logs/cubepick-reach_xxxx.json

The report puts the run status, configuration, metrics, scene results, and recorded policy conversations on one page. Agent notes from tool calls are highlighted above their call lines. For runs captured with --store-frames, the report also embeds the stored camera frames at the exact observation turns where the model saw them. The file contains its stylesheet and frame data inline and uses native browser controls to collapse transcripts, so it has no network or JavaScript dependency.

By default, view replaces the log path's suffix with .html and prints the written path. Use -o REPORT.html to choose another file, -o - to write only the HTML document to stdout, or --open to launch the written file in the default browser. Missing output directories are created. The command returns 0 whenever it produces the report, even when the evaluation recorded a failed or cancelled run.

Frame embedding is on by default when the log's frame directory can be found. Use --no-frames to keep the transcript placeholders, or --frames-budget MB to change the default 50 MB inline-frame payload limit. --frames-budget 0 removes the limit. Inlined frames make the HTML document larger, so use a smaller budget or --no-frames when page size matters.

inspect-robots video

Render a --store-frames run's stored camera frames into one MP4 per (trial, camera) stream:

inspect-robots video logs/adhoc_xxxx.json

fps: 10 (control_hz from log)
wrote logs/frames/20260715_184213/scene-0-e0_left_cam.mp4 (1200 frames)
wrote logs/frames/20260715_184213/scene-0-e0_right_cam.mp4 (1200 frames)
wrote 2/2 streams

Encoding is done by the ffmpeg binary (no Python dependencies are added); install it from your package manager, or point at a specific build with --ffmpeg PATH. Videos land in the frames directory by default (--out DIR overrides). The playback rate defaults to the log's control_hz and can be overridden with --fps N. A stream that fails to encode is reported on stderr and the remaining streams still encode; the exit code is 1 if any stream failed.

inspect-robots --version

inspect-robots --version