Skip to main content

API reference

Generated automatically from the source docstrings. The public, stability-guaranteed surface is everything exported by inspect_robots.__all__ (eval, eval_set, read_eval_log, EvalLog and the other log dataclasses); the sections below document the full framework.

Core types & spaces​

inspect_robots.types​

Core observation/action data types exchanged between policy and embodiment.

These are the wire format of a rollout. They are deliberately small, immutable, and NumPy-native. Arrays are raw (the policy owns model-specific preprocessing); images are (H, W, C) uint8.

The dataclasses set eq=False because they carry NumPy arrays, whose element-wise == does not yield a single bool — identity/round-trip semantics are what callers actually need here.

ImageArray​

ImageArray = npt.NDArray[np.uint8]

StateArray​

StateArray = npt.NDArray[np.floating[Any]]

Observation​

class Observation(images: Mapping[str, ImageArray] = dict(), state: Mapping[str, StateArray] = dict(), instruction: str | None = None, image_times: Mapping[str, float] = dict(), state_time: float = 0.0, extra: Mapping[str, Any] = dict()):

A single multi-modal observation produced by an embodiment.

images are keyed by camera name; state holds proprioception keyed by a controlled vocabulary (e.g. "eef_pos", "gripper"). instruction is the language goal for this step (usually constant across an episode, but may change for long-horizon tasks). The rollout injects the current step into the policy-facing observation's extra["env_step"] and reserves that key; embodiments should not set it.

Action​

class Action(data: StateArray, meta: Mapping[str, Any] = dict()):

A single action to apply to an embodiment.

Semantics (control mode, rotation representation, gripper kind, frame) live on the action space, not on every action instance — see inspect_robots.spaces.

ActionChunk​

class ActionChunk(actions: Sequence[Action], control_hz: float | None = None, inference_latency_s: float | None = None, meta: Mapping[str, Any] = dict()):

A horizon of actions predicted by one policy inference.

Modern VLAs (π0, ACT, diffusion policies) predict H future actions that are executed open-loop because inference is slower than the control rate. H == 1 is the degenerate "reactive policy" case. control_hz is the rate the chunk was intended to be played at — advisory metadata a consumer may honor (the rollout enforces no rate; None leaves it unspecified). inference_latency_s, when measured, is logged.

OPERATOR_END​

OPERATOR_END = 'operator_end'

StepResult​

class StepResult(observation: Observation, reward: float | None = None, terminated: bool = False, termination_reason: str | None = None, truncated: bool = False, info: Mapping[str, Any] = dict()):

The outcome of applying one action to an embodiment.

terminated means the task ended (success or hard failure); termination_reason disambiguates (e.g. "success", "collision", "fault", "out_of_bounds", "operator_end" — the standard reason for operator-ended episodes awaiting a verdict). truncated means a time/horizon cutoff. A simulator may expose privileged success via info.

inspect_robots.spaces​

Action/observation spaces and action semantics.

Spaces describe the shape of actions and observations; ActionSemantics describes what an action means (control mode, rotation representation, gripper kind, reference frame). Semantics are what make compatibility checking real (a 7-DoF VLA vs a 6-DoF arm; delta vs absolute poses) and make temporal ensembling correct.

This module ships a minimal-but-functional core for the tracer slice; richer validation and the full StateSpec vocabulary are layered on in a later step without changing these signatures.

ControlMode​

ControlMode = Literal['joint_pos', 'joint_vel', 'joint_delta', 'eef_delta_pose', 'eef_abs_pose', 'eef_delta_pos']

RotationRepr​

RotationRepr = Literal['none', 'quat_wxyz', 'quat_xyzw', 'rot6d', 'axis_angle', 'euler_xyz']

GripperKind​

GripperKind = Literal['none', 'continuous', 'binary']

Frame​

Frame = Literal['base', 'world', 'camera']

ABSOLUTE_CONTROL_MODES​

ABSOLUTE_CONTROL_MODES = frozenset({'joint_pos', 'eef_abs_pose'})

ActionSemantics​

class ActionSemantics(control_mode: ControlMode, rotation_repr: RotationRepr = 'none', gripper: GripperKind = 'none', frame: Frame = 'base', dim_labels: tuple[str, ...] | None = None, max_step: tuple[float | None, ...] | None = None):

What an action vector means. Attached to an action Box.

dim_labels optionally names each action dimension (e.g. ("left_j0", ..., "right_gripper") for a bimanual arm) so tooling — LLM agent policies, logging, visualization — can address dimensions by name. Length is validated against the owning box in Box.__post_init__ (the semantics/shape pairing is only visible there).

max_step optionally declares the safe per-control-step change for each absolute-target dimension in the action space's native units. A None entry leaves that dimension to range-based defaults. Displacement and rate boxes already declare per-step limits through their bounds, so declarations are rejected for those modes.

Box​

class Box(shape: tuple[int, ...], low: npt.NDArray[np.floating[Any]] | None = None, high: npt.NDArray[np.floating[Any]] | None = None, semantics: ActionSemantics | None = None):

A continuous box-shaped space. Optional low/high bounds and, for action spaces, ActionSemantics.

CameraSpec​

class CameraSpec(name: str, height: int, width: int, channels: int = 3):

An image stream an embodiment provides or a policy requires.

CANONICAL_STATE_UNITS​

CANONICAL_STATE_UNITS: dict[str, str] = {'joint_pos': 'rad', 'joint_vel': 'rad/s', 'eef_pos': 'm', 'eef_pose': 'm+quat', 'eef_quat': 'unit_quat', 'gripper': 'normalized', 'gripper_width': 'm'}

StateField​

class StateField(key: str, shape: tuple[int, ...], unit: str = '', dtype: str = 'float64'):

One proprioception field: its key, shape, unit, and dtype.

StateSpec​

class StateSpec(fields: tuple[StateField, ...] = ()):

A richer description of an embodiment's proprioception than a bare key set.

ObservationSpace​

class ObservationSpace(cameras: tuple[CameraSpec, ...] = (), state_keys: frozenset[str] = frozenset(), state: StateSpec | None = None):

The observations an embodiment provides / a policy requires.

state_keys is the compatibility-relevant set of proprioception keys. state optionally carries the richer StateSpec (shapes/units).

Policy & embodiment​

inspect_robots.policy​

The Policy (VLA) interface — one of Inspect Robots's two swappable inputs.

A Policy is the "brain": given an Observation (plus the scene's instruction), it returns an ActionChunk to be executed open-loop.

The public contract is a runtime-checkable Policy Protocol so callers can wrap existing models without inheriting. PolicyBase is an optional convenience ABC with sane defaults.

PolicyConfig​

class PolicyConfig(action_horizon: int = 1, replan_interval: int | None = None, temperature: float | None = None):

Inference-time configuration, recorded in the eval log.

The VLA analog of Inspect's GenerateConfig: action-chunk handling and sampling knobs that affect reproducibility.

PolicyInfo​

class PolicyInfo(name: str, action_space: Box, observation_space: ObservationSpace = ObservationSpace(), control_hz: float | None = None):

Static description of a policy used for compatibility checking + logging.

Policy​

class Policy(Protocol):

The VLA contract.

Policies may additionally define five optional hooks, none part of this Protocol so existing policies stay conformant. bind(embodiment_info) lets embodiment-adaptive policies adopt the embodiment's spaces; eval() calls it after resolving both components and before compatibility checking. on_trial_start(scene_id, epoch, log_dir, run_id) runs immediately before each trial's rollout. It is a policy lifecycle hook, distinct from the sink bus hook of the same name, which runs first and takes only scene id and epoch. on_trial_end(record, log_dir, run_id) runs when a trial finishes (including errored and cancelled trials, except trials whose on_trial_start raised, which never reached reset()), before sinks see the record. Mutations to record.metadata land in the log, and exceptions degrade the run to status="error" rather than crashing the overall evaluation. transcript() returns a small JSON-serializable audit record for the current trial, such as an LLM conversation. The framework calls it once per trial at trial end after a successful reset(), including errored trials. It must be idempotent and safe between resets, must not mutate policy state, and its return value must not alias live state. Camera images must not be embedded because frame sidecars already persist them. Collection runs on the rollout thread and is best-effort: the framework normalizes and bounds the result, and a raising or misbehaving hook cannot change trial outcome. transcript_delta() returns plain-JSON-type messages appended since its previous call, or since reset() on the first call, and returns None or an empty list when nothing is new. Implementations must sanitize only the new slice in O(new messages), including eliding image bytes before the result reaches visualization sinks, and reset() must rewind the cursor. PolicyBase ships defaults for bind() and transcript() but deliberately has no transcript_delta() default: policies must opt in so every inference does not pay for a no-op hook call.

PolicyBase​

class PolicyBase(ABC):

Optional base class providing defaults; inherit only for the helpers.

inspect_robots.embodiment​

The Embodiment interface — Inspect Robots's second swappable input.

An Embodiment is the "body + world": a real robot or a simulator. It produces observations, executes actions, and owns the action/observation spaces, the native control rate, and reset/safety machinery.

Designed around real-robot reality: reset may drive to a home pose and block on human confirmation; there is no guaranteed privileged success oracle. Simulators are a stricter special case that opt into extra capabilities.

Per R1 (see the design doc): rollout() applies no wall-clock pacing of its own — step() is called as fast as the policy/controller/approver can produce actions. An embodiment that needs real-time cadence (e.g. matching a real robot's control period) paces itself inside step(), and declares the "self_paced" capability to document that it does.

Capability​

Capability = str

SEEDABLE​

SEEDABLE: Capability = 'seedable'

RESETTABLE​

RESETTABLE: Capability = 'resettable'

AUTO_RESET​

AUTO_RESET: Capability = 'auto_reset'

PRIVILEGED_SUCCESS​

PRIVILEGED_SUCCESS: Capability = 'privileged_success'

RENDERABLE​

RENDERABLE: Capability = 'renderable'

SELF_PACED​

SELF_PACED: Capability = 'self_paced'

EmbodimentInfo​

class EmbodimentInfo(name: str, action_space: Box, observation_space: ObservationSpace, control_hz: float | None = None, is_simulated: bool = False, capabilities: frozenset[Capability] = frozenset(), supported_setups: frozenset[str] = frozenset(), supported_target_kinds: frozenset[str] = frozenset(), docs: str | None = None):

Static description of an embodiment for compatibility checking + logging.

docs contains free-form markdown operating notes for policies that can read text, such as joint layout and positive directions, zero-pose geometry, gripper polarity, frame conventions, and workspace hints. Keep it concise because consumers inject it into system prompts verbatim. None means no notes are offered; consumers must treat empty and whitespace-only strings the same as absence.

Embodiment​

class Embodiment(Protocol):

The robot/simulator contract.

Embodiments may additionally define an optional bind_task(envelope) hook (not part of this Protocol, so existing embodiments stay conformant): after compatibility checking, eval() calls it with the task's resolved TaskEnvelope before the first reset, letting adapters learn the rollout horizon — e.g. an operator countdown showing elapsed/total. A seconds-based task is converted to integer steps using info.control_hz before the hook fires. The hook is optional input, not a guarantee: it never fires on direct rollout() calls or on older cores, so adapters must keep a graceful fallback. It fires once per eval(), which can be several times over an embodiment's lifetime; each call replaces the previous envelope. EmbodimentBase ships a no-op default.

Embodiments may also define an optional contribute_guardrails(self, action_space: Box) -> GuardrailContribution hook, likewise omitted from this Protocol so existing implementations remain runtime-conformant. The CLI calls it with the resolved action space and appends its contribution after the generic clamp and delta limiter. Direct rollout() and programmatic eval() callers compose their own approvers, so the hook is CLI input rather than a universal callback. EmbodimentBase returns an empty contribution by default.

EmbodimentBase​

class EmbodimentBase(ABC):

Optional base class with a no-op close; inherit for the convenience.

User configuration​

inspect_robots.defaults​

Read the user configuration shared by Inspect Robots and plugin CLIs.

Component names from environment variables override names from <config-home>/inspect-robots/config.ini, but they never override the args owners recorded by that file. An args section is valid only for its owner — apply embodiment_args only after confirming the owner is the embodiment you drive, or the args recorded for someone else's rig will configure yours:

defaults = load_defaults(os.environ)
if defaults.embodiment_args_owner == "my_embodiment":
    args = defaults.embodiment_args

(Plugins with several embodiments can resolve the owner through inspect_robots.registry.registered and check its class instead of comparing names.)

A missing file yields empty defaults. A malformed or type-invalid file raises SystemExit naming the file, with a plain one-line message that callers may catch.

Defaults​

class Defaults(policy: str | None = None, policy_source: str | None = None, embodiment: str | None = None, embodiment_source: str | None = None, sim_embodiment: str | None = None, sim_embodiment_source: str | None = None, scorer: str | None = None, max_steps: int | None = None, store_frames: bool = False, rerun: bool = False, policy_args: dict[str, Any] = dict(), embodiment_args: dict[str, Any] = dict(), sim_embodiment_args: dict[str, Any] = dict(), policy_args_owner: str | None = None, embodiment_args_owner: str | None = None, sim_embodiment_args_owner: str | None = None):

Resolved user defaults, each with a human-readable source for the run header.

config_path​

def config_path(env: Mapping[str, str]) -> Path | None:

Return the user config file path derived from env.

The result is <config-home>/inspect-robots/config.ini, whether or not the file exists. Return None when neither XDG_CONFIG_HOME nor HOME is set; a variable set to the empty string counts as unset.

load_defaults​

def load_defaults(env: Mapping[str, str]) -> Defaults:

Load user defaults: environment variables override the config file.

env is injected (pass os.environ) so tests never touch the real home directory. Environment variables override component names without changing the config file's args owners. A missing config file yields empty defaults; a malformed or type-invalid one raises SystemExit naming the file. The exception carries a plain one-line message and is catchable by diagnostic callers that need to continue other checks.

Tasks & scenes​

inspect_robots.scene​

Scenes — the robotics analog of Inspect AI's Sample.

A Scene is one initial condition of a benchmark: a language instruction, an optional success Target, an optional seed, and metadata. A benchmark Task iterates over a dataset of scenes (e.g. 50 object layouts), repeated epochs times.

Field mapping to Inspect: Sample(input, target, id, metadata, setup) ↔ Scene(instruction, target, id, metadata, setup, init_seed).

Target​

class Target(kind: str, spec: Mapping[str, Any] = dict()):

A success specification the scorer reads. Embodiment-namespaced.

kind names what the embodiment must realize/evaluate (e.g. "reach_object"); spec carries the parameters. Kept intentionally open for the tracer; richer typed targets land with the scorer milestone.

Scene​

class Scene(id: str, instruction: str, target: Target | None = None, init_seed: int | None = None, setup: str | None = None, metadata: Mapping[str, Any] = dict()):

One initial condition of a benchmark.

ListSceneDataset​

class ListSceneDataset(scenes: Sequence[Scene]):

A trivial in-memory scene dataset backed by a sequence.

inspect_robots.task​

The Task — an embodiment-agnostic benchmark definition.

Mirrors Inspect AI's Task = dataset + scorer + epochs/reducer, adapted for robotics: the dataset is a sequence of Scene initial conditions and the rollout horizon (max_steps) lives here.

Epochs​

class Epochs(count: int = 1, reducer: str = 'mean'):

Repeat count plus the reducer used to combine per-epoch scores.

Mirrors Inspect's Epochs(count, reducer); reducer is a registered name (default "mean").

TaskEnvelope​

class TaskEnvelope(name: str, max_steps: int):

Identity and rollout limits of a task, safe to hand to adapters.

This is what eval() passes to an embodiment's optional bind_task(envelope) hook: enough for the adapter to display or pre-allocate for the run (e.g. an operator countdown against max_steps), and nothing that would let it second-guess scoring or the dataset. Deliberately carries no control rate — the rollout enforces no wall-clock rate of its own (R1, revised); a self-paced embodiment owns its own cadence.

Task​

class Task(name: str, scenes: Sequence[Scene], scorer: Scorer | str | Sequence[Scorer | str], max_steps: int | None = None, epochs: int | Epochs = 1, metadata: Mapping[str, Any] = dict(), *, max_seconds: float | None = None):

A benchmark: scenes + scorer(s) + horizon, independent of any embodiment.

scorer accepts scorer objects or registry names (e.g. scorer="success_at_end"), or a sequence mixing both.

Declare exactly one rollout horizon. max_steps is already resolved; max_seconds is keyword-only and is resolved against the paired embodiment's positive finite control_hz by eval().

Scoring​

inspect_robots.scorer​

Scoring: Scores, the Scorer protocol, epoch reducers, and builtin scorers.

Mirrors Inspect AI's @scorer/reducer split. A scorer maps a recorded trajectory (+ the scene's Target) to a Score; an epoch reducer collapses the per-epoch scores of one scene into a single score before metrics aggregate across scenes.

Scorers consume the recorded trajectory (not a live environment), so scoring is reproducible from a saved log.

ScoreValue​

ScoreValue = bool | int | float | str

Reducer​

Reducer = Callable[[Sequence['Score']], 'Score']

Score​

class Score(value: ScoreValue, explanation: str | None = None, metadata: Mapping[str, Any] = dict()):

The outcome a scorer assigns to one trajectory.

value_to_float​

def value_to_float(value: ScoreValue) -> float:

Coerce a score value to a float for metric aggregation.

Scorer​

class Scorer(Protocol):

Maps a recorded trajectory + scene target to a Score.

reduce_mean​

def reduce_mean(scores: Sequence[Score]) -> Score:

Collapse numeric epoch values with the arithmetic mean.

reduce_median​

def reduce_median(scores: Sequence[Score]) -> Score:

Collapse numeric epoch values with the median.

reduce_max​

def reduce_max(scores: Sequence[Score]) -> Score:

Keep the largest numeric epoch value.

reduce_min​

def reduce_min(scores: Sequence[Score]) -> Score:

Keep the smallest numeric epoch value.

reduce_mode​

def reduce_mode(scores: Sequence[Score]) -> Score:

Most common raw value (works for categorical scores). Deterministic.

pass_at_k​

def pass_at_k(k: int) -> Reducer:

Unbiased pass@k estimator over the epoch scores (success = value >= 0.5).

get_reducer​

def get_reducer(name: str) -> Reducer:

Resolve a builtin reducer or parse a dynamic pass_at_<k> name.

reduce_scores​

def reduce_scores(name: str, scores: Sequence[Score]) -> Score:

Apply the named epoch reducer to one scene's scores.

success_at_end​

def success_at_end() -> Scorer:

Score 1.0 iff the episode terminated with reason "success".

episode_length​

def episode_length() -> Scorer:

Score = number of environment steps taken.

min_distance_to_goal​

def min_distance_to_goal() -> Scorer:

Score = the closest the effector got to the goal (lower is better).

reached_goal_state​

def reached_goal_state(threshold: float = 0.05) -> Scorer:

Success iff the effector came within threshold of the goal.

operator_scorer​

def operator_scorer() -> Scorer:

Score from the human operator's recorded success judgement (R6).

VLMScorer​

class VLMScorer:

Reserved interface (R10): score from a VLM classifier over final frames.

Implemented in a later milestone; instantiating and calling it raises so the contract is visible but no half-baked behavior ships.

Rollout, controllers & safety​

inspect_robots.rollout​

The rollout engine — the closed control loop at the heart of Inspect Robots.

One rollout runs a single trial (one scene, one epoch): it drives the policy↔embodiment loop through the Controller (open-loop chunk execution) and the Approver safety gate, logging each step to the sinks, and returns an immutable TrialRecord that scorers consume.

derive_seed​

def derive_seed(eval_seed: int | None, scene_seed: int | None, epoch: int) -> int:

Deterministically combine eval/scene seeds and the epoch index (R2).

Distinct epochs of the same scene get distinct seeds so repeats actually vary for stochastic policies, while a fixed (eval_seed, scene_seed, epoch) reproduces bitwise. None and 0 hash differently, so an unseeded input does not silently alias seed=0.

StepRecord​

class StepRecord(t: int, observation: Observation, action: Action, result: StepResult, image_refs: Mapping[str, FrameRef] | None = None):

One step of a recorded trajectory.

When a FrameStore is used, observation has its images stripped and image_refs holds on-disk handles instead (R5).

TrialRecord​

class TrialRecord(scene_id: str, epoch: int, seed: int | None, steps: list[StepRecord] = list(), terminated: bool = False, truncated: bool = False, termination_reason: str | None = None, status: str = 'success', error: str | None = None, inference_latencies: list[float] = list(), operator_judgement: str | None = None, operator_note: str | None = None, events: list[Event] = list(), metadata: dict[str, Any] = dict(), policy_transcript: Any = None):

The full record of one trial — the unit scorers consume.

rollout​

def rollout(policy: Policy, embodiment: Embodiment, scene: Scene, *, max_steps: int, seed: int | None, epoch: int, controller: Controller, approver: Approver, sink: LogSink, frame_store: FrameStore | None = None) -> TrialRecord:

Run a single trial and return its record.

Generic exceptions raised by the policy are wrapped as PolicyError; by the embodiment as EmbodimentFault; by the approver as SafetyAbort (an approver that crashed cannot vouch for safety). Already-typed Inspect Robots errors (incl. SafetyAbort) propagate unchanged, so the eval orchestrator can apply the correct continue-vs-halt policy. Every error raised from inside the trial carries the partial TrialRecord on exc.record for the orchestrator to preserve.

The loop applies no wall-clock pacing of its own: embodiment.step() is called as fast as the policy/controller/approver can produce actions. An embodiment that needs real-time cadence paces itself inside step() and declares the "self_paced" capability to document that it does (see Embodiment).

inspect_robots.controller​

Controllers — the rollout middleware layer (Inspect's @solver analog).

A Controller owns the per-control-step decision of which action to send to the embodiment. It internally decides when to call policy.act() (a slow VLA inference returning an ActionChunk), buffers the returned chunk, and pops the next action each step. This single-method, stateful shape (R3) is what lets advanced controllers — e.g. a temporal-ensembling controller that re-infers every step and blends overlapping predictions — compose without forking the rollout loop.

DefaultController plays the first replan_interval actions of each chunk, then re-infers (replan_interval=None ⇒ play the whole chunk before replanning).

Controller​

class Controller(Protocol):

Decides the next action to execute, calling the policy as needed.

DefaultController​

class DefaultController(replan_interval: int | None = None):

Open-loop chunk execution with periodic replanning.

SmoothingController​

class SmoothingController(inner: Controller, alpha: float = 0.5):

Wrap another controller and exponentially smooth its action stream.

Demonstrates the middleware composition the single-method interface enables: the wrapped controller owns inference/replanning while this layer applies an exponential moving average (alpha toward the new action) on top. Only valid for additive/continuous action spaces (the caller's responsibility).

EnsemblingController​

class EnsemblingController(action_space: Box, m: float = 0.1):

ACT/ALOHA-style temporal ensembling over overlapping action chunks.

Queries the policy every control step and blends, for the current step, the predictions of all still-relevant recent chunks. A chunk queried at global step q predicts step t via its action at index t - q (valid while 0 <= t - q < len(chunk)). Predictions are weighted exp(-m * i) with i = 0 for the oldest contributing chunk (ALOHA's convention: older predictions dominate, which smooths motion); larger m favors the oldest.

Only valid for additive action representations: the constructor refuses rotation reps and binary grippers that cannot be linearly averaged (R8).

inspect_robots.approver​

The Approver — a safety gate between policy output and the embodiment.

Every action passes through Approver.review before embodiment.step. This is the robotics analog of Inspect AI's ApprovalPolicy and is more safety-critical: an approver may pass, clamp, or veto an action (a veto raises SafetyAbort). In the tracer slice the default approver passes everything through; clamping/operator approval land in rollout hardening.

Approver​

class Approver(Protocol):

Reviews an action before it reaches the embodiment.

May return the action unchanged, return a modified (e.g. clamped) action, or raise SafetyAbort to halt the eval.

GuardrailContribution​

class GuardrailContribution(approvers: tuple[tuple[str, Approver], ...] = (), warnings: tuple[str, ...] = ()):

Approvers an embodiment adds to the CLI's default guardrail chain.

approvers pairs a short display name (shown in the guardrails: banner) with each approver to append. warnings names contributions the embodiment declined to make or made in a degraded state, and why, so neither condition is invisible. Warnings may accompany active approvers.

A contributed approver must preserve the incoming Action object's identity when approving it unmodified. If it substitutes another target, such as holding an earlier pose, it must call DeltaLimitApprover.rewind_reference with that executed pose so later deltas are measured from reality. It vetoes by raising SafetyAbort; other exceptions are bugs and propagate. It must validate its own input, rejecting non-finite values with SafetyAbort rather than assuming an upstream clamp exists, because it may be the only active gate.

AutoApprover​

class AutoApprover:

Approve every action unchanged (the permissive default).

ClampApprover​

class ClampApprover(action_space: Box):

Clamp actions to a box's low/high bounds before they reach hardware.

One-sided boxes are honored: a low-only box clamps from below, a high-only box from above. A modified action is flagged via action.meta["clamped"] so the rollout can record an approval event; when nothing clamps, the same action object is returned (the rollout detects modification by identity).

Non-finite values are the safety cases: NaN anywhere in the action raises SafetyAbort — a NaN is a poisonous value with no meaningful clamp, and it must never reach hardware. ±inf is not an abort: it clamps to the finite bound on that side like any other out-of-range value (and passes through if that side is unbounded).

DeltaLimitApprover​

class DeltaLimitApprover(action_space: Box, max_delta: float | Any | None = None):

The "no wild swings" gate — semantics-aware per-step change limiting.

Absolute-target modes (joint_pos, eef_abs_pose) clamp each dimension to at most max_delta away from the last approved action of the trial; the first action passes through un-delta-limited (there is no trustworthy reference yet — bounds clamping still applies upstream). The derived default is a declared ActionSemantics.max_step where present, otherwise 5% of high - low per step.

Displacement/rate modes (eef_delta_pos, eef_delta_pose, joint_delta, joint_vel) are the per-step change, so each dimension clamps to the intersection of the box and [-max_delta, +max_delta]. The derived default is the box alone — core cannot assume displacement bounds are per-step-sized, so without an explicit max_delta the limiter adds nothing beyond ClampApprover.

Construction never guesses: missing semantics, a needed missing/non-finite bound without an explicit max_delta, an absolute pose mode whose rotation representation cannot be clamped per-dimension, or a displacement pose mode carrying a quaternion or rot6d delta (whose identity is not the zero vector, so per-dimension clamping distorts it) all raise ValueError. NaN anywhere in a reviewed action raises SafetyAbort. A modified action is flagged meta["delta_clamped"]; an unmodified one is returned as the same object (rollout detects modification by identity). The reference lives in the rollout store (fresh per trial) under a namespaced key.

ChainApprover​

class ChainApprover(*approvers: Approver):

Run approvers in sequence, feeding each the previous one's result.

Gives "guardrails" one composable name, e.g. ChainApprover(ClampApprover(space), DeltaLimitApprover(space)). Identity is preserved end to end: if no approver modifies the action, the original object comes back, so the rollout's modification check still works.

inspect_robots.frames​

FrameStore — rollout-owned streaming of camera frames to disk (R5).

A long multi-camera episode would exhaust memory if every frame were retained in the TrialRecord. Instead the rollout streams frames to disk through a FrameStore and keeps only lightweight FrameRef handles. This is owned by the rollout, NOT by any log sink, so trajectories are recorded (and scorable) independent of which optional sinks are enabled.

FrameRef​

class FrameRef(camera: str, t: int, path: str):

A handle to a camera frame stored on disk.

FrameStore​

class FrameStore(root: str):

Persist frames as .npy files under root and hand back refs.

inspect_robots.transcript​

A typed transcript of rollout events.

Each trial records an ordered stream of events (reset, inference, step, approval, operator judgement, error). This is the robotics analog of Inspect AI's transcript and is the data a results viewer renders. Events are deliberately lightweight: a kind, the step index t (-1 for pre-loop events), and a small data payload.

EventKind​

EventKind = str

Event​

class Event(kind: EventKind, t: int, data: Mapping[str, Any] = dict()):

One entry in a trial's transcript.

reset_event​

def reset_event(seed: int | None) -> Event:

Record the seed used before control step zero.

inference_event​

def inference_event(t: int, latency_s: float | None, chunk_len: int) -> Event:

Record one policy call's timing and buffered action count at step t.

step_event​

def step_event(t: int, terminated: bool, truncated: bool, reason: str | None) -> Event:

Record termination state after executing control step t.

approval_event​

def approval_event(t: int, modified: bool, detail: str | None = None) -> Event:

Record whether the safety gate changed the proposed action at step t.

operator_event​

def operator_event(t: int, verdict: str, source: str = 'prompt', note: str | None = None) -> Event:

Record the operator's verdict, optional note, and source after the rollout ends.

verdict carries whatever the operator answered, which includes the non-verdict "skip" when they declined to grade a trial but still left a note. Consumers deriving an outcome from this event must treat "skip" as "no judgement", never as a result.

error_event​

def error_event(t: int, error_type: str, message: str) -> Event:

Record an exception's type and message at the failing control step.

Compatibility & errors​

inspect_robots.compat​

Compatibility checking between a policy and an embodiment.

Before any rollout, Inspect Robots verifies that a (policy, embodiment) pair can actually run together: the action spaces agree in dimension and semantics, the embodiment provides every observation the policy requires (resolving a name remap), the control rates are reconcilable (R1), and — given a task — every scene is realizable on the embodiment (R7).

Hard mismatches are error issues that fail fast; soft ones are warnings.

CompatIssue​

class CompatIssue(severity: str, code: str, message: str):

One compatibility finding with a stable code and human-readable detail.

CompatibilityReport​

class CompatibilityReport(issues: list[CompatIssue] = list(), remap: dict[str, str] = dict()):

The outcome of a compatibility check.

check_compatibility​

def check_compatibility(policy: Policy, embodiment: Embodiment, task: Task | None = None, *, remap: dict[str, str] | None = None) -> CompatibilityReport:

Return a structured compatibility report (does not raise).

assert_compatible​

def assert_compatible(policy: Policy, embodiment: Embodiment, task: Task | None = None, *, remap: dict[str, str] | None = None) -> CompatibilityReport:

Check compatibility and raise CompatibilityError on hard errors.

inspect_robots.errors​

Inspect Robots error taxonomy.

The split below resolves the "fail fast vs never-crash-overnight" tension:

  • ConfigError / CompatibilityError are raised before any rollout — bad configuration should fail loudly and immediately.
  • PolicyError is recorded as a failed trial; whether it aborts the eval is governed by fail_on_error (Inspect semantics).
  • EmbodimentFault and SafetyAbort always halt the eval regardless of fail_on_error — a faulted or unsafe robot must never auto-advance to the next scene unattended.

InspectRobotsError​

class InspectRobotsError(Exception):

Base class for all Inspect Robots errors.

When an error is raised from inside a running trial, the rollout engine attaches the partial TrialRecord — the steps walked and the transcript events up to the failure — as record, so the orchestrator can preserve it in logs. record is None for errors raised outside a rollout (configuration, compatibility, ...).

ConfigError​

class ConfigError(InspectRobotsError):

Invalid task / policy / embodiment configuration. Fail fast.

CompatibilityError​

class CompatibilityError(InspectRobotsError):

A policy and embodiment are not compatible. Fail fast, before any rollout.

PolicyError​

class PolicyError(InspectRobotsError):

The policy raised during inference. Recorded as a failed trial.

Connection-level failures carry a remediation hint in the message.

EmbodimentFault​

class EmbodimentFault(InspectRobotsError):

The embodiment/robot faulted. Always halts the eval and requires a human.

SafetyAbort​

class SafetyAbort(InspectRobotsError):

An approver vetoed an action / e-stop. Always halts the eval.

Adapter conformance​

inspect_robots.conformance​

Adapter conformance: mechanical checks on an embodiment's declared spaces.

Plan 0008 made two consumers of declarations load-bearing: the CLI's default guardrails derive their limits from the action space, and the LLM agent policy builds its whole tool surface from the spaces at bind time. An adapter with missing semantics, missing bounds, unlabeled dims, or a misaligned StateSpec silently degrades both. This module turns those requirements into a checkable report so adapter repos can enforce them in CI (one test: assert_embodiment_conformant(MyEmbodiment().info)) and users can audit an installed adapter via inspect-robots doctor.

check_embodiment is purely declarative and touches no hardware. The separate, opt-in check_guardrail_contribution executes plugin code to validate the optional runtime contribution hook. Conformance proves an adapter is guardrail-ready and agent-ready, not that its declarations are honest (a delta rig declaring absolute-sized per-step bounds type-checks fine). The adapter authoring guide covers the human half.

DEVICE_KINDS​

DEVICE_KINDS = ('v4l2', 'can', 'serial')

DeviceSlot​

class DeviceSlot(arg: str, kind: str, label: str, group: str | None = None):

One device-shaped constructor argument the setup wizard interviews.

arg is the [embodiment.args] key to write; kind selects the probe ("v4l2": /dev/v4l listings with unplug-identify; "can": SocketCAN netdevs from sysfs with unplug-identify; "serial": /dev/serial/by-id listing). label is the human prompt ("left arm CAN channel"). Slots sharing a non-None group are all-or-none: the wizard refuses to write a partial subset of the group.

device_slots​

def device_slots(factory: object) -> tuple[DeviceSlot, ...]:

The declared device slots, defensively read.

Reads DEVICE_SLOTS off factory; anything that is not an iterable of DeviceSlot instances (or contains a slot whose kind is not recognized) has the offending entries ignored, never crashes the wizard. Returns a tuple in declaration order.

OptionSlot​

class OptionSlot(arg: str, label: str, default: bool = False):

One boolean behavior toggle the setup wizard interviews.

arg is the [embodiment.args] key to write (true/false); label is the yes/no question shown to the operator ("Skip the operator start prompts (auto_start)"); default is the suggested answer when the key is absent from an existing config.

option_slots​

def option_slots(factory: object) -> tuple[OptionSlot, ...]:

The declared option slots, defensively read.

Reads OPTION_SLOTS off factory; anything that is not an iterable of OptionSlot instances has the offending entries ignored, never crashes the wizard. Returns a tuple in declaration order.

missing_runtime_requirements​

def missing_runtime_requirements(factory: object) -> dict[str, str]:

The declared runtime modules that are not importable here.

Reads RUNTIME_REQUIREMENTS (module name -> remediation command) off factory and probes each with importlib.util.find_spec. Top-level names (the intended use) are probed without executing anything; a dotted name imports its parent package when present, so declare top-level names. ANY probe failure counts as missing (broad except Exception: a present-but-broken parent package propagates arbitrary errors from its __init__, and this checker must never crash setup or doctor). Entries whose key or value is not str, or a RUNTIME_REQUIREMENTS that is not a Mapping, are ignored (a plugin typo must not crash the preflight). Returns the missing subset, insertion-ordered.

ConformanceIssue​

class ConformanceIssue(severity: str, code: str, message: str):

One finding: severity is "error" (fails the check) or "warning".

ConformanceReport​

class ConformanceReport(embodiment: str, issues: tuple[ConformanceIssue, ...] = tuple()):

All findings for one embodiment; ok iff there are no errors.

check_embodiment​

def check_embodiment(info: EmbodimentInfo) -> ConformanceReport:

Check an embodiment's declarations against the plan-0008 requirements.

Errors: missing action semantics; missing/non-finite bounds; missing or duplicate dim_labels; absolute-target modes without exactly one StateSpec field shaped like the action space; a space the default guardrail chain refuses to limit. Warnings: control_hz undeclared (agent motion falls back to 10 Hz step counting) and zero-width bound dims. Purely declarative — safe to run anywhere, no hardware touched.

assert_embodiment_conformant​

def assert_embodiment_conformant(info: EmbodimentInfo) -> None:

Pytest-friendly wrapper: raise AssertionError with the full summary.

check_guardrail_contribution​

def check_guardrail_contribution(embodiment: Embodiment, action_space: Box) -> ConformanceReport:

Validate an optional contribution hook by executing plugin code.

An absent attribute passes. A present attribute must be callable and must return GuardrailContribution. Exceptions raised by the hook propagate as plugin bugs. Unlike check_embodiment, callers must opt in knowing this check can run arbitrary adapter code.

assert_guardrail_contribution_conformant​

def assert_guardrail_contribution_conformant(embodiment: Embodiment, action_space: Box) -> None:

Raise AssertionError with the full contribution report on failure.

Evaluation & logs​

inspect_robots.eval​

The eval() entry point — orchestrates scenes x epochs into an EvalLog.

Mirrors Inspect AI's eval(): it runs a task's scenes (repeated over epochs), scores each recorded trajectory, reduces epochs, aggregates metrics, and returns a list of immutable EvalLog (one per task). The tracer slice accepts already-constructed objects; registry-string resolution (policy="openvla/7b") is layered on with the registry milestone.

eval​

def eval(task: Task | str, policy: Policy | str, embodiment: Embodiment | str, *, log_dir: str = 'logs', sinks: list[LogSink] | None = None, seed: int | None = 0, fail_on_error: bool | float = False, controller: Controller | None = None, approver: Approver | None = None, remap: dict[str, str] | None = None, store_frames: bool = False, before_scoring: Callable[[TrialRecord, Scene], None] | None = None) -> list[EvalLog]:

Run task with policy on embodiment; return [EvalLog].

task/policy/embodiment may be objects or registry names (e.g. policy="scripted"), resolved through the registry — the Inspect-style ergonomic that keeps logs and the CLI reproducible. An embodiment resolved from a registry name is owned by eval() and is closed when the run finishes (even on a halt); a caller-constructed embodiment object stays open — the caller owns its lifecycle.

seed=None draws a fresh seed from the OS and records it in the log, so an "unseeded" run remains reproducible after the fact (and is distinct from seed=0).

fail_on_error follows Inspect semantics for PolicyError (True = fail on first, False = never, 0<x<1 = proportion, x>1 = count), checked after every trial. EmbodimentFault/SafetyAbort always halt regardless. Errored trials are recorded (with their partial trajectory delivered to sinks) but never scored, so a failed trial cannot masquerade as data in the metrics; it stays visible via SceneResult.status and an empty entry in SceneResult.epochs.

A run in which every trial errored (nothing was scored) always ends with status == "error", regardless of fail_on_error.

Ctrl-C during a rollout records the partial trial and writes a log with status == "cancelled", then re-raises the interrupt (as a KeyboardInterrupt subclass chaining the original) after on_eval_end completes. An interrupt outside the rollout call (during scoring, reducers, or log assembly), or a second interrupt during the cancellation handlers, may still prevent the log from being written.

When store_frames is set, camera frames are streamed to <log_dir>/frames as binary side-cars (R5) rather than kept in memory.

before_scoring is called exactly once per trial that will be scored (never for errored trials, which are recorded but not scored), after the rollout returns and before the scorers run. It may mutate the record — e.g. capture TrialRecord.operator_judgement (R6) so the operator scorer can read it, and TrialRecord.operator_note alongside it, which is recorded but never scored. Exceptions it raises propagate to the caller. Note this fires on the other side of scoring from LogSink.on_trial_end.

Raises CompatibilityError (fail fast, before any rollout) if the policy and embodiment are incompatible, and ConfigError for an invalid epoch reducer.

eval_set​

def eval_set(tasks: Task | str | Sequence[Task | str], policy: Policy | str, embodiment: Embodiment | str, *, log_dir: str = 'logs', seed: int | None = 0, fail_on_error: bool | float = False, controller: Controller | None = None, approver: Approver | None = None, remap: dict[str, str] | None = None, store_frames: bool = False, before_scoring: Callable[[TrialRecord, Scene], None] | None = None, retry_attempts: int = 0) -> tuple[bool, list[EvalLog]]:

Run a set of tasks and return (success, logs) (mirrors Inspect AI).

success is True iff every task's log has status == "success".

Resumption of a partially-completed run (skipping already-finished scenes via a stable run id) is reserved for a follow-up: retry_attempts is accepted now so callers don't get retrofitted, but is not yet honored.

inspect_robots.log​

The immutable evaluation log — Inspect Robots's reproducible record of a run.

Mirrors Inspect AI's EvalLog: version + status + eval spec + results + stats + per-scene samples + error. Serialized to JSON with a schema version so newer Inspect Robots always reads older logs (a read-back guarantee enforced by golden tests in a later step).

Immutability is shallow: the dataclasses are frozen and sequence fields are tuples, so reassigning a field or mutating the sample list is impossible — but dict-valued fields (SceneResult.reduced, the per-epoch score dicts, EvalResults.metrics, EvalSpec.policy_config / embodiment_info) remain plain mutable dicts, and SceneResult.policy_transcripts entries are arbitrary mutable JSON values. Treat a log as read-only; nothing deep-freezes it.

SCHEMA_VERSION​

SCHEMA_VERSION = 1

EvalSpec​

class EvalSpec(task: str, policy: str, embodiment: str, created: str, inspect_robots_version: str, git_commit: str | None = None, policy_config: dict[str, Any] = dict(), embodiment_info: dict[str, Any] = dict(), seed: int | None = None, max_steps: int | None = None, max_seconds: float | None = None):

Top-level identity and configured horizon of a reproducible eval.

max_steps is always the resolved integer budget used by the rollout. max_seconds preserves a benchmark's declared physical-time budget when that integer was derived from the embodiment's control rate.

EvalStats​

class EvalStats(started_at: str, completed_at: str, duration_s: float, total_steps: int, mean_inference_latency_s: float | None = None, frames_dir: str | None = None):

Timing and execution statistics for a run.

SceneResult​

class SceneResult(scene_id: str, status: str, reduced: dict[str, float] = dict(), epochs: tuple[dict[str, float], ...] = (), error: str | None = None, instruction: str | None = None, operator_judgements: tuple[str | None, ...] = (), operator_notes: tuple[str | None, ...] = (), trial_metadata: tuple[dict[str, Any], ...] = (), termination_reasons: tuple[str | None, ...] = (), policy_transcripts: tuple[Any, ...] = ()):

Per-scene result: the reduced score(s) plus the raw per-epoch scores.

EvalResults​

class EvalResults(total_scenes: int, total_trials: int, metrics: dict[str, float] = dict(), errored_trials: int = 0):

Aggregate results across all scenes.

EvalLog​

class EvalLog(version: int, status: str, eval: EvalSpec, results: EvalResults, stats: EvalStats, samples: tuple[SceneResult, ...] = (), error: str | None = None):

The full record returned by eval and persisted to disk.

read_eval_log​

def read_eval_log(path: str) -> EvalLog:

Read an EvalLog back from a JSON file on disk.

Logging sinks​

inspect_robots.logging.sink​

The LogSink protocol, its optional extension, and a no-op base implementation.

A sink observes a run's lifecycle. The rollout engine and eval() call these hooks in a fixed order: on_eval_start → (per trial: on_trial_start → log_step* → on_trial_end) → on_eval_end.

Sinks may additionally define the duck-typed log_policy_messages(t, messages) extension. The rollout calls it at most once per control step and only when the policy performed an inference. Policy implementations are expected to supply plain-JSON-type messages shaped like TrialRecord.policy_transcript entries, but core does not enforce or normalize that shape on this live path, so sinks must render defensively. Sinks must not mutate the supplied messages. The extension deliberately stays off both LogSink and NullSink: it must not change structural protocol conformance or advertise a no-op that makes policies build transcript deltas.

LogSink​

class LogSink(Protocol):

Observes the lifecycle of an evaluation run.

NullSink​

class NullSink:

A sink that does nothing — a convenient base for partial implementations.

inspect_robots.logging.json_log​

The canonical JSON eval-log sink.

Writes the immutable EvalLog to log_dir once the run finishes. The write is atomic (temp file + os.replace) so an interrupted overnight run never leaves a half-written log.

The file is strict RFC 8259 JSON: non-finite floats (nan, ±inf, e.g. a min_distance_to_goal score when no distance was ever recorded) are mapped to null before serialization, so any conforming parser (jq, browsers, non-Python tooling) can read the log. allow_nan=False is kept on the json.dump call as a regression backstop: if a non-finite value ever slips past the sanitizer, writing fails loudly instead of emitting Infinity/ NaN literals.

JsonLogSink​

class JsonLogSink(log_dir: str):

Persist the final EvalLog as JSON.

Per-step data lives in the TrialRecord/FrameStore, not here; this sink only writes the final log (path holds where it landed).

inspect_robots.logging.rerun_sink​

Optional Rerun visualization sink.

Logs camera images, proprioception, action vectors, and success markers to Rerun <https://github.com/rerun-io/rerun>_. The sink can write a .rrd recording, spawn a local viewer, or connect over gRPC to a remote viewer. A viewer spawned by the sink has a 2 GiB memory limit by default, which makes the viewer purge its oldest events instead of accumulating an unbounded session history. rerun-sdk is imported lazily inside methods so the core package never depends on it; if it is not installed, the sink warns once and becomes a no-op (so unattended runs and the core-only import gate are unaffected).

Emission happens on a daemon worker thread: log_step snapshots the transition and enqueues it, so a slow or stalled viewer connection can never block the control-rate rollout loop. Under backpressure the sink degrades visualization instead of delaying control: camera frames are dropped first (scalar plots stay complete), then whole steps or transcript rows, and the drop counts are reported as a RuntimeWarning when the eval ends. The queue is drained at every trial boundary (bounded by flush_timeout), so an eval that aborts mid-run loses at most the current trial's queued tail. Camera frames are JPEG-compressed by default (jpeg_quality=75); pass jpeg_quality=None for lossless raw frames. If compression is unavailable (an SDK without Image.compress, or pillow missing), the sink warns once and logs raw frames. All Rerun SDK calls after init/spawn/connect_grpc/save happen on the worker because the SDK's timeline state is thread-local, except for the shutdown-path flush probe and unregister_shutdown on the caller path. The probe invokes RecordingStream.flush on a bounded daemon thread; flush is internally synchronized, and unregister_shutdown only manipulates an atexit hook, so neither depends on timeline state. Worker state is generation-scoped so a worker wedged in a blocked SDK call is disowned at shutdown and can never double-consume after a restart. If the SDK flush probe also wedges, the sink unregisters the SDK's unbounded atexit flush, disables itself, and abandons queued SDK-side data.

The viewer limit applies only to viewers this package spawns; a viewer already running on the default port keeps the limit it started with. The bounded exit probe requires rerun-sdk 0.22 or newer because older recording streams expose no flush method. A new sink in the same process can still hang in rr.init after a connection wedges, and paths such as Ctrl-C that skip on_eval_end retain the SDK's unbounded atexit hook.

Each trial's entities are namespaced under trial/<scene_id>/e<epoch> so successive trials never overwrite one another on the shared step timeline. Transcripts are emitted as TextLog rows at {prefix}/llm paired with a markdown TextDocument at {prefix}/llm/latest holding the step's assistant message(s) for a wrapped, timeline-synced reading pane.

Install with pip install "inspect-robots[rerun]".

RerunSink​

class RerunSink(recording_path: str | None = None, *, application_id: str = 'inspect_robots', spawn: bool = False, spawn_memory_limit: str = '2GiB', connect_url: str | None = None, jpeg_quality: int | None = 75, queue_size: int = 64, flush_timeout: float = 10.0):

Write a .rrd, spawn a bounded local viewer, or connect to a remote one.

spawn_memory_limit is passed verbatim to Rerun only when spawn=True.

Registry & CLI​

inspect_robots.registry​

Registry and decorators for tasks, policies, embodiments, scorers, and sinks.

Mirrors Inspect AI's extension model: components register by name via decorators and are resolved from strings (so eval(policy="scripted") and the CLI work). Out-of-tree packages publish components through importlib.metadata entry-point groups, so an installed inspect-robots-openvla appears in inspect-robots list without being imported first.

Entry-point groups: inspect_robots.tasks, inspect_robots.policies, inspect_robots.embodiments, inspect_robots.scorers, inspect_robots.sinks.

Kind​

Kind = str

KINDS​

KINDS: tuple[Kind, ...] = ('task', 'policy', 'embodiment', 'scorer', 'sink')

F​

F = TypeVar('F', bound=(Callable[..., Any]))

register​

def register(kind: Kind, name: str | None = None) -> Callable[[F], F]:

Register a factory under kind/name (defaults to its __name__).

task​

def task(name: str | None = None) -> Callable[[F], F]:

Decorator: register a task factory under name.

policy​

def policy(name: str | None = None) -> Callable[[F], F]:

Decorator: register a policy factory under name.

embodiment​

def embodiment(name: str | None = None) -> Callable[[F], F]:

Decorator: register an embodiment factory under name.

scorer​

def scorer(name: str | None = None) -> Callable[[F], F]:

Decorator: register a scorer factory under name.

sink​

def sink(name: str | None = None) -> Callable[[F], F]:

Decorator: register a log-sink factory under name.

registered​

def registered(kind: Kind) -> dict[str, Callable[..., Any]]:

Return all registered factories for kind (builtins + plugins).

resolve​

def resolve(kind: Kind, name: str, /, **kwargs: Any) -> Any:

Construct a registered component by name with the given keyword args.

inspect_robots.cli​

The inspect_robots command-line interface.

Subcommands:

  • inspect-robots list [tasks|policies|embodiments|scorers|sinks] — show registered components (builtins + installed plugins).
  • inspect-robots run --task T --policy P --embodiment E — run an eval, resolving components from the registry. Pass constructor args with -T/-P/-E k=v; --epochs, --fail-on-error, and --store-frames tune the run. The written log's path is printed at the end.
  • inspect-robots eval-set TASK [TASK ...] --policy P --embodiment E — run several registered tasks (exact names or fnmatch globs, e.g. 'kitchenbench/*') against one resolved policy/embodiment pair via eval_set. Prints one status line and a compact per-task row instead of a full summary per task.
  • inspect-robots inspect LOG.json [--transcript] [--wire [CALL]] — print a saved eval log and optionally append policy conversations or captured wire calls.
  • inspect-robots summarize LOG.json [--model M] — distill a saved eval log into a deterministic digest or model-written learnings file.
  • inspect-robots view LOG.json|LOG_DIR [-o PATH] [--open] [--serve] — render one saved eval log or a browsable directory index as self-contained HTML, optionally serving a directory until stopped.
  • inspect-robots video LOG.json — render a --store-frames run's stored camera frames to one MP4 per (trial, camera) stream via the ffmpeg binary.
  • inspect-robots setup — interactively configure defaults and camera devices.

Zero-config form (plan 0005): inspect-robots "place the spoon on the plate" is sugar for run --instruction "..." — a single ad-hoc scene on the user's default policy/embodiment (flags > INSPECT_ROBOTS_POLICY/_EMBODIMENT env vars > ~/.config/inspect-robots/config.ini). The sugar only fires for a first argument with interior whitespace, so a mistyped subcommand (inspect-robots isnpect) errors instead of starting a robot rollout; single-word instructions use the explicit run --instruction form.

DEFAULT_RERUN_CONNECT_URL​

DEFAULT_RERUN_CONNECT_URL = 'rerun+http://127.0.0.1:9876/proxy'

build_parser​

def build_parser() -> argparse.ArgumentParser:

Build the command-line parser and its subcommands.

main​

def main(argv: Sequence[str] | None = None) -> int:

Parse arguments, dispatch one subcommand, and return its process exit code.

Mock world​

inspect_robots.mock.cubepick​

CubePick — a deterministic 2D toy world for exercising the full stack.

A point end-effector in the unit square must reach a cube. The action is a 2D end-effector position delta. Success is declared (and exposed as privileged info["success"]) when the effector is within goal_radius of the cube. Fully deterministic given a seed; no third-party dependencies.

CubePickEmbodiment​

class CubePickEmbodiment(*, max_step: float = 0.1, goal_radius: float = 0.05, start: tuple[float, float] = (0.1, 0.1)):

A 2D reach-the-cube simulator.

inspect_robots.mock.policies​

Mock policies for the CubePick world.

  • ScriptedPolicy — a deterministic oracle that walks the effector to the cube. It predicts a full action chunk by simulating its own future motion, so the chunk is a genuine open-loop trajectory (H > 1).
  • RandomPolicy — emits random deltas; mostly fails.
  • NoopPolicy — emits zero actions; never succeeds.

ScriptedPolicy​

class ScriptedPolicy(*, chunk_size: int = 4, max_step: float = 0.1):

Deterministic oracle: walk straight to the cube, in chunks.

RandomPolicy​

class RandomPolicy(*, chunk_size: int = 4, max_step: float = 0.1, seed: int = 0):

Emit random small deltas. Deterministic given the construction seed.

NoopPolicy​

class NoopPolicy(*, chunk_size: int = 1):

Emit zero actions; never moves.