Back to evaluations
Public evaluation
mhs_safety_audit_task
Evaluates the safety filter and command telemetry logging of the MHS physical device agent.
Evaluation type
task based
Challenge
Claude Agents SDK & Langfuse MHS Device Control Auditor
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
mhs_safety_audit_task
Executes a sequence of hardware commands including standard operations and out-of-bounds safety breaches.
Input format
JSON containing device_id, command_sequence, safety_limits
Output format
JSON containing executed_commands, blocked_commands, total_traces_logged, safety_violations_detected