Multimodal Cobot Quality Inspection and Safety Monitoring with Google ADK
Explosion-proof painting cobots operating in hazardous spray environments require combined visual quality inspection and worker safety distance checking. Build a multimodal monitoring assistant using Google ADK and Gemini Multimodal to detect paint thickness coverage defects and worker zone violations in real time.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a multimodal inspection agent using Google ADK and Gemini to inspect paint finishes and enforce safety zone boundaries around robotic paint arms.
How work is evaluated
Evaluates defect detection precision and cobot pause signal latency upon human zone breach.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Safety Pause Verification
Ensures action_command is set to TRIGGER_EMERGENCY_PAUSE when human is in zone.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
defect_detection_accuracy
Accuracy score on paint coating thickness classification. • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Construct a Google ADK agent integrating Gemini multimodal vision capabilities
Process camera frame inputs to verify coating coverage and detect paint runs or thin spots
Trigger instantaneous safety pause commands if human operators breach Class I Div 1 boundary zones
Reference links and supporting material
Multimodal inspection records and simulated camera telemetry from explosion-proof paint cobot cells.
How this agent runs
Evaluates defect detection precision and cobot pause signal latency upon human zone breach.
Challenge input
JSON containing frame_id, coating_thickness_um, human_detected_in_zone
Google ADK
Provides seamless multimodal tool integration and Vertex AI infrastructure support.
Evaluated output
JSON with quality_pass, defect_type, action_command
- Ensures action_command is set to TRIGGER_EMERGENCY_PAUSE when human is in zone.
- Accuracy score on paint coating thickness classification. • target: 0.95 • range: 0-1
- Benchmark: Multimodal Robotic Safety Benchmark
- Defect Detection Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- Google · Required
- Google ADK · Optional
- Google Jules · Optional
- Google Jules · Optional
Evaluation contract
- Safety Pause Verification · Weight 1
- defect_detection_accuracy · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.