Back to evaluations
Public evaluation

integrator_case_solver

Assess precision and recall of multi-agent answers against expert solutions for CSIA integrator troubleshooting cases.

Evaluation type
task based
Challenge
Multi-Agent Integrator Knowledge Assistant with AutoGen and Qwen 3
Difficulty
Advanced
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
1
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

integrator_case_solver

Solve industrial communication and PLC configuration error cases

Input format

JSON containing error_code, controller_type, and symptom_description

Output format

JSON with diagnostic_step, solution_code, and manual_reference