Back to evaluations
Public evaluation
integrator_case_solver
Assess precision and recall of multi-agent answers against expert solutions for CSIA integrator troubleshooting cases.
Evaluation type
task based
Challenge
Multi-Agent Integrator Knowledge Assistant with AutoGen and Qwen 3
Difficulty
Advanced
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
integrator_case_solver
Solve industrial communication and PLC configuration error cases
Input format
JSON containing error_code, controller_type, and symptom_description
Output format
JSON with diagnostic_step, solution_code, and manual_reference