Public evaluation

DataIngestionAndCleanup

Evaluation focuses on the system's ability to successfully process a complex, multi-source dataset, perform accurate schema mapping, and integrate the cleaned data into a target system, demonstrating data quality and automation efficiency.

Evaluation type

task based

Challenge

Automate GenAI Data Prep & System Integration

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

DataIngestionAndCleanup

Agents must ingest mock customer data from a CSV and an API, identify duplicates and missing values, and produce a cleaned, normalized output.

Input format

{'csv_path': 'string', 'api_endpoint': 'string'}

Output format

{'cleaned_data_json': [{'customer_id': 'int', 'name': 'string', 'email': 'string', 'address': 'string'}]}

Task 2

SchemaMappingAndIntegration

Agents must map the cleaned customer data schema to a target CRM system's schema (provided as a JSON definition) and produce a valid payload for integration via a mock MCP-enabled CRM API.

Input format

{'cleaned_data_json': [...], 'crm_schema_json': {'properties': {'id': {'type': 'number'}, 'contact_name': {'type': 'string'}} }}

Output format

{'integration_payloads': [{'id': 1, 'contact_name': 'Alice Smith', 'contact_email': 'alice@example.com'}]}