CostOptimizedModelTraining
Participants will submit a containerized (Docker) agent system designed to orchestrate a series of simulated AI compute workloads across a multi-cloud environment. Evaluation will focus on the system's ability to minimize cost, successfully complete tasks, and adhere to security and efficiency parameters.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
CostOptimizedModelTraining
Orchestrate a simulated model training job for a specified dataset across AWS and a simulated Azure/GCP, minimizing overall compute cost while meeting a deadline. Agents must dynamically choose instance types and regions.
{ 'scenario_id': 'string', 'model_size_gb': int, 'dataset_size_gb': int, 'required_gpu_hours': int, 'deadline_hours': int, 'max_budget_usd': float }
{ 'cloud_allocations': [{'cloud': 'AWS'|'Azure', 'instance_type': 'string', 'region': 'string', 'gpu_hours': int, 'cost_usd': float}], 'total_cost_usd': float, 'completion_time_hours': float, 'explanation': 'string', 'agent_communication_log': [] }
DistributedDataProcessing
Process a large distributed dataset across cloud storage (S3, Blob Storage), requiring agents to provision compute, transfer data securely, and coordinate processing tasks.
{ 'scenario_id': 'string', 'data_location_aws': 's3://bucket/path', 'data_location_azure': 'https://storageaccount.blob.core.windows.net/container/path', 'processing_task_description': 'string', 'security_level': 'high'|'medium' }
{ 'processing_status': 'completed'|'failed', 'result_location_aws': 's3://bucket/output', 'processing_logs_summary': 'string', 'security_audit_report': 'string', 'agent_communication_log': [] }