Back to evaluations
Public evaluation

DetectSuspiciousTransaction

The evaluation will assess the agent's accuracy in detecting predefined suspicious financial activities, the relevance and comprehensiveness of its compliance reports, and the efficiency of its tool usage.

Evaluation type
task based
Challenge
Autonomous Crypto Compliance Agent
Difficulty
Advanced
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
2
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

DetectSuspiciousTransaction

Evaluate the agent's ability to identify a specific type of illicit financial activity (e.g., money laundering pattern) from a stream of simulated transaction data.

Input format

{ "transaction_stream": [ { "id": "T101", "sender": "A", "receiver": "B", "amount": 1000, "timestamp": "..." }, ... ], "regulatory_context": "[relevant Pinecone query result]" }

Output format

{ "is_suspicious": true, "reason": "[explanation]", "flagged_transactions": ["T105", "T106"], "compliance_breach": "[rule name]", "recommendations": ["action"] }

Task 2

GenerateComplianceReport

Assess the clarity, comprehensiveness, and accuracy of the compliance report generated by the agent for a given suspicious event.

Input format

{ "suspicious_event_details": { "type": "structuring", "flagged_transactions": [...] }, "relevant_regulations": "[Pinecone query result]" }

Output format

{ "report_summary": "[text]", "evidence_links": ["url"], "risk_assessment": "High" }