Skip to main content
Back to challenges
Challenge

Autonomous Predictive Vision & Repair with Claude Agents and Cognition Devin

Industrial vision systems are critical for identifying wear and tear in harsh factory conditions (dust, heat, vibration). This challenge tasks you with creating an agentic workflow using the Claude Agents SDK to process visual sensor data and Cognition Devin to autonomously generate corrective code for the PLC (Programmable Logic Controller). You will build an agent that uses Claude's vision capabilities to detect 'thermal bloom' or 'bearing misalignment' from simulated industrial camera frames. Once a defect is detected, the Claude agent must use the 'Computer Use' capability to navigate a simulated IDE where Cognition Devin is active. Devin will then be tasked with writing a Python-based predictive maintenance script that adjusts the machine's operating frequency to mitigate further damage until a human technician arrives. This addresses the transition of industrial automation into a 'software discipline' and the move toward 'agentic AI' in modern manufacturing.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a multi-agent pipeline where Claude Agents detect visual anomalies and Cognition Devin generates PLC-level corrective scripts.

Delivery guide

How work is evaluated

Evaluation

Evaluates if the vision agent correctly identifies a misaligned bearing and if Devin generates a valid frequency-reduction script.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Frequency Bound Check

Ensures the generated script doesn't set frequency to zero or unsafe levels.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Vision Accuracy

Precision in identifying the defect coordinates in the image • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement visual anomaly detection using Claude's multi-modal agent capabilities

  • Orchestrate a complex task delegation between Claude and Cognition Devin

  • Develop predictive maintenance logic based on visual and sensor fusion

  • Design an automated PLC script generation and verification pipeline

Resources and assets

Reference links and supporting material

Dataset notes

A collection of 50 synthetic thermal and RGB images of industrial motors, some showing clear misalignment or overheating, coupled with machine metadata.

Challenge setup

Evaluates if the vision agent correctly identifies a misaligned bearing and if Devin generates a valid frequency-reduction script.

Challenge input

PNG (Thermal Image) + Current Machine Specs

Claude Agents SDK

Superior multi-modal reasoning for visual fault analysis.

Cognition Devin

Capable of end-to-end software generation for control systems.

Evaluated output

Python Script + Validation Report

Evaluation checks
  • Ensures the generated script doesn't set frequency to zero or unsafe levels.
  • Precision in identifying the defect coordinates in the image • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: RoboMaintenance-v1
  • Vision Accuracy target: 0.9
  • Protected reference cases support private evaluation
Configured runtime
  • Docker execution harness
  • Python sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Claude Code · Required
Observation
  • Claude Code · Required
Policy Serving
  • Claude 4.1 Opus · Optional
  • Claude 4 Sonnet · Optional

Evaluation contract

  • Frequency Bound Check · Weight 1
  • Vision Accuracy · Weight 1

Metrics

  • Vision Accuracy · Target 0.9

Benchmarks

  • RoboMaintenance-v1
  • robotics

Reference cases

  • Protected reference cases are configured.

Workflow tasks

  1. Task 1

    Input: PNG (Thermal Image) + Current Machine Specs

    Output: Python Script + Validation Report

    Execution harness: Docker

Evaluation tools

  • Claude Agents SDK · Vision Reasoning
  • Cognition Devin · Autonomous Coding

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Autonomous Predictive Vision & Repair with Claude Agents and Cognition Devin