Product_Research_and_Comparison
The evaluation will assess the system's ability to accurately research products, make reasonable purchasing decisions, and demonstrate robust tool integration and agent collaboration. Key metrics will include factual accuracy, decision logic, and efficiency.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
Product_Research_and_Comparison
Agent system researches three competing products for a given user need and outputs a comparison table.
{'user_need': 'string', 'budget': 'int'}
{'comparison_table': [{'product_name': 'string', 'features': ['string'], 'price': 'float', 'rating': 'float'}], 'recommended_product': 'string', 'reasoning': 'string'}
Simulated_Purchase_Decision
Agent system simulates the process of selecting a product and initiating a purchase, demonstrating Model Context Protocol tool usage.
{'product_id': 'string', 'quantity': 'int', 'shipping_address': 'string'}
{'order_status': 'string', 'payment_gateway_response': 'json', 'estimated_delivery': 'string'}