FraudDetectionAccuracy
The agent will be evaluated on its ability to accurately detect simulated fraud events in a stream of transaction data, minimize false positives, and deliver timely alerts with relevant context.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
FraudDetectionAccuracy
Given a stream of simulated crypto transactions, the agent must identify known fraud patterns (e.g., large conversions to Monero after a social engineering scam event) and trigger an alert.
{'transaction_stream': [{'tx_id': '...', 'from_addr': '...', 'to_addr': '...', 'amount': '...', 'currency': '...', 'timestamp': '...'}, ...], 'news_feed': [{'headline': '...', 'content': '...', 'timestamp': '...'}, ...]}
{'alerts': [{'type': 'fraud', 'detected_pattern': '...', 'suspicious_txs': ['tx_id_1', ...], 'context': '...', 'timestamp': '...'}]}
FalsePositiveRate
Ensure the agent does not trigger alerts for normal, legitimate high-volume transactions.
{'transaction_stream': [...], 'news_feed': [...]}
{'alerts': []}