Automate GenAI Data Prep & System Integration
IT industry is adapting to GenAI by focusing on data cleanup and system integration. This challenge aims to build an autonomous multi-agent system using AutoGen, powered by Gemini 3 Pro, to automate complex data preparation and schema mapping for enterprise AI adoption. Agents will collaborate to ingest raw data, perform transformations, reconcile schemas across disparate enterprise systems, and integrate cleaned data using MCP-enabled tools. The system will employ hybrid reasoning, combining Gemini's advanced data understanding with structured data processing tools, to ensure data quality and seamless integration, thereby accelerating enterprise GenAI readiness.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an AutoGen multi-agent system with Gemini 3 Pro to automate data preparation, schema mapping, and system integration for GenAI readiness using MCP toolchains.
How work is evaluated
Evaluation focuses on the system's ability to successfully process a complex, multi-source dataset, perform accurate schema mapping, and integrate the cleaned data into a target system, demonstrating data quality and automation efficiency.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Design an AutoGen multi-agent system for end-to-end data preparation
Integrate enterprise systems via MCP-enabled tools for data ingestion and export
Implement advanced schema mapping and reconciliation using Gemini 2.5 Pro
Utilize RAG techniques with LlamaIndex for context-aware data transformation
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
Evaluation focuses on the system's ability to successfully process a complex, multi-source dataset, perform accurate schema mapping, and integrate the cleaned data into a target system, demonstrating data quality and...
Challenge input
{'csv_path': 'string', 'api_endpoint': 'string'}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'cleaned_data_json': [{'customer_id': 'int', 'name': 'string', 'email': 'string', 'address': 'string'}]}
- Output data must meet specified quality metrics (e.g., >95% completeness, no duplicates).
- All relevant fields must be correctly mapped to the target schema.
- TransformationAccuracy target: 0.95
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Output data must meet specified quality metrics (e.g., >95% completeness, no duplicates).
- All relevant fields must be correctly mapped to the target schema.
Metrics
- TransformationAccuracy · Target 0.95
- IntegrationPayloadValidity · Target 0.98
Workflow tasks
- Task 1
Input: {'csv_path': 'string', 'api_endpoint': 'string'}
Output: {'cleaned_data_json': [{'customer_id': 'int', 'name': 'string', 'email': 'string', 'address': 'string'}]}
Execution harness: Python
- Task 2
Input: {'cleaned_data_json': [...], 'crm_schema_json': {'properties': {'id': {'type': 'number'}, 'contact_name': {'type': 'string'}} }}
Output: {'integration_payloads': [{'id': 1, 'contact_name': 'Alice Smith', 'contact_email': 'alice@example.com'}]}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.