Skip to main content
Challenge

Claude Agents SDK Model Hardware Standard Device Enclosure & Inspector

Build a hardware-enclosed agent containment system using Claude Agents SDK to validate physical actuator instructions against Model Hardware Standard specifications. Implement real-time hardware telemetry monitoring and safety sandboxing to guarantee safe operation with sub-10ms emergency brake latency.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an isolated hardware execution sandbox and MHS inspector tool using Claude Agents SDK to control physical computing interfaces safely.

Delivery guide

How work is evaluated

Evaluation

Evaluates response latency and blocking efficacy of MHS hardware containment hooks.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

voltage_interception_test

Confirms over-voltage request was blocked.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

emergency_brake_latency_ms

Latency in triggering safety interlock. • target: 6 • range: 0-20

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement Claude Agents SDK containment wrapper around physical hardware device APIs

  • Build real-time MHS protocol inspection interceptors checking voltage, amperage, and mechanical motion bounds

  • Enforce hardware sandbox boundary isolation isolating LLM tool outputs from direct memory-mapped I/O

  • Achieve sub-10ms emergency brake trigger latency upon safety condition breach

Resources and assets

Reference links and supporting material

Dataset notes

Telemetry suite containing 300 hardware signal frames spanning safe and out-of-spec physical device operations.

Challenge setup

Evaluates response latency and blocking efficacy of MHS hardware containment hooks.

Challenge input

JSON object with target device, target voltage, and safety threshold parameters.

Claude Agents SDK

Anthropic official framework for agent construction with tool safety boundaries.

Evaluated output

JSON containment result indicating INTERCEPTED status and emergency break response time in ms.

Evaluation checks
  • Confirms over-voltage request was blocked.
  • Latency in triggering safety interlock. • target: 6 • range: 0-20
Configured evaluation evidence
  • Benchmark: Hardware Sandbox Safety Benchmark
  • Emergency Brake Latency Ms target: 6
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Claude Code · Required
  • Anthropic · Optional
Observation
  • Claude Code · Required
Policy Serving
  • Claude 4.1 Opus · Optional

Evaluation contract

  • voltage_interception_test · Weight 1
  • emergency_brake_latency_ms · Weight 1

Metrics

  • emergency_brake_latency_ms · Target 6

Benchmarks

  • Hardware Sandbox Safety Benchmark
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with target device, target voltage, and safety threshold parameters.

    Output: JSON containment result indicating INTERCEPTED status and emergency break response time in ms.

    Execution harness: Python

Evaluation tools

  • Claude Agents SDK · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Claude Agents SDK Model Hardware Standard Device Enclosure & Inspector