SimulatedClientVetting
The evaluation will focus on the agent system's ability to accurately and securely execute a series of simulated financial tasks, adhere to compliance rules, and demonstrate robustness under varying conditions. Ax will be used to track success rates and decision quality.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
SimulatedClientVetting
Evaluate the agent's capability to process a client onboarding request, verify provided information against a mock database, and flag potential compliance issues.
{'client_id': 'string', 'name': 'string', 'address': 'string', 'employment': 'string', 'financial_history': 'list'}
{'status': 'string', 'compliance_flags': 'list', 'summary': 'string'}
SimulatedTradeExecution
Assess the agent's ability to execute a hypothetical stock trade given market conditions and risk parameters, using a mock trading API and Groq Cloud for price analysis.
{'trade_id': 'string', 'symbol': 'string', 'quantity': 'integer', 'max_price': 'float', 'risk_tolerance': 'string'}
{'status': 'string', 'executed_price': 'float', 'message': 'string'}