Skip to main content
Loading navigation
Back to challenges

Multi-Agent Integrator Knowledge Assistant with AutoGen and Qwen 3

Control system integrators lose hundreds of hours searching legacy technical manuals during commissioning. Build a multi-agent system using AutoGen and Qwen 3 to synthesize field manuals and troubleshoot PLC/SCADA integration errors. Achieve over 90% precision on industrial troubleshooting queries.

EngineeringHosted by Vera
Sign in free to start

What you are building

Construct an AutoGen multi-agent team powered by Qwen 3 to solve complex system integrator technical queries and hardware faults.

How work is evaluated

Evaluation

Assess precision and recall of multi-agent answers against expert solutions for CSIA integrator troubleshooting cases.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

correct_manual_citation

Cites appropriate hardware reference manual

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

solution_precision

Precision of recommended troubleshooting actions • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Build an AutoGen multi-agent setup (User Proxy, Technical Search Agent, PLC Expert Agent)

  • Incorporate Qwen 3 for high-accuracy multilingual industrial reasoning

  • Integrate vector search tools for hardware manual documentation retrieval

  • Evaluate multi-agent convergence on complex commissioning scenarios

Resources

Dataset notes

Collection of CSIA integrator troubleshooting cases across Ethernet/IP, Modbus TCP, and Profinet error logs.

Challenge setup

Assess precision and recall of multi-agent answers against expert solutions for CSIA integrator troubleshooting cases.

Challenge input

JSON containing error_code, controller_type, and symptom_description

AutoGen

Enables multi-agent conversational decision workflows.

Qwen 3

Provides underlying domain reasoning for technical manuals.

Evaluated output

JSON with diagnostic_step, solution_code, and manual_reference

Evaluation checks
  • Cites appropriate hardware reference manual
  • Precision of recommended troubleshooting actions • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: Industrial System Integrator QA Benchmark
  • Solution Precision target: 0.9
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Alibaba Cloud (Qwen) · Optional
Policy Serving
  • Qwen 3 · Optional

Evaluation contract

  • correct_manual_citation · Weight 1
  • solution_precision · Weight 1

Metrics

  • solution_precision · Target 0.9

Benchmarks

  • Industrial System Integrator QA Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing error_code, controller_type, and symptom_description

    Output: JSON with diagnostic_step, solution_code, and manual_reference

    Execution harness: Python

Evaluation tools

  • AutoGen · Multi-Agent Framework
  • Qwen 3 · Language Model

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Multi-Agent Integrator Knowledge Assistant with AutoGen and Qwen 3