Automate GenAI Data Prep & System Integration
IT industry is adapting to GenAI by focusing on data cleanup and system integration. This challenge aims to build an autonomous multi-agent system using AutoGen, powered by Gemini 3 Pro, to automate complex data preparation and schema mapping for enterprise AI adoption. Agents will collaborate to ingest raw data, perform transformations, reconcile schemas across disparate enterprise systems, and integrate cleaned data using MCP-enabled tools. The system will employ hybrid reasoning, combining Gemini's advanced data understanding with structured data processing tools, to ensure data quality and seamless integration, thereby accelerating enterprise GenAI readiness.
What you are building
The core problem, expected build, and operating context for this challenge.
IT industry is adapting to GenAI by focusing on data cleanup and system integration. This challenge aims to build an autonomous multi-agent system using AutoGen, powered by Gemini 3 Pro, to automate complex data preparation and schema mapping for enterprise AI adoption. Agents will collaborate to ingest raw data, perform transformations, reconcile schemas across disparate enterprise systems, and integrate cleaned data using MCP-enabled tools. The system will employ hybrid reasoning, combining Gemini's advanced data understanding with structured data processing tools, to ensure data quality and seamless integration, thereby accelerating enterprise GenAI readiness.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
What you should walk away with
Orchestrate AutoGen role-based agent teams (e.g., Data Engineer Agent, Schema Mapper Agent, Quality Assurance Agent) for collaborative data pipeline execution
Leverage Gemini 3 Pro's multi-modal capabilities for understanding diverse data formats (e.g., spreadsheets, PDFs, JSON schemas) and generating transformation logic
Implement MCP-enabled tool integration with enterprise data sources (e.g., mock CRM/ERP APIs, SQL databases) for automated data extraction and loading
Build dynamic schema mapping agents that use LlamaIndex for RAG over enterprise documentation and data dictionaries to intelligently reconcile disparate schemas
Develop hybrid reasoning workflows where Gemini 2.5 Pro identifies data quality issues and generates Python scripts (via its code generation capabilities) for rectification, executed by a Code Executor Agent
Design graph-based data lineage tracking and transformation workflows to visualize data flow and dependencies across agents and systems
Implement self-correcting mechanisms where agents can detect and resolve data inconsistencies or integration failures autonomously
How this agent runs
Evaluation focuses on the system's ability to successfully process a complex, multi-source dataset, perform accurate schema mapping, and integrate the cleaned data into a target system, demonstrating data quality and...
Challenge input
{'csv_path': 'string', 'api_endpoint': 'string'}
Agent execution
The configured agent processes the input under the challenge policy.
Evaluated output
{'cleaned_data_json': [{'customer_id': 'int', 'name': 'string', 'email': 'string', 'address': 'string'}]}
- Output data must meet specified quality metrics (e.g., >95% completeness, no duplicates).
- All relevant fields must be correctly mapped to the target schema.
- TransformationAccuracy target: 0.95
- Python execution harness
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- The evaluation module defines the checks.
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset and AI stack
Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.
Scope a managed run[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.