Real-Time Fraud Alert Triage Engine with Google ADK and Gemini Multimodal OCR
Financial fraud investigators face high triage fatigue, sorting through thousands of low-probability false alerts daily. Modern fraud detection requires combining real-time transaction telemetry with visual context, such as identity documents, receipts, and physical check scans. In this challenge, you will implement a multimodal fraud alert triage engine using Google's Agent Development Kit (ADK) and Gemini 1.5 Flash/Pro. The agent will ingest transaction payloads along with image evidence, perform automated OCR and anomaly checks, and assign an evidence-backed alert priority score.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a multimodal fraud triage agent with Google ADK and Gemini to parse visual and transactional evidence.
How work is evaluated
Evaluates accuracy of visual text extraction and fraud classification decision accuracy.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
mismatch_detection
Verifies that amount discrepancies are correctly identified
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
fraud_triage_accuracy
Accuracy score on test set containing mixed valid and fraudulent alerts • target: 94 • range: 0-100
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Initialize Google ADK agents with Vertex AI or Gemini API backends
Process multimodal inputs combining structured JSON logs and image payloads
Extract invoice/check details using Gemini OCR capabilities to verify against transaction values
Route high-confidence true positives directly to tier-2 investigators with generated evidence briefs
Reference links and supporting material
Synthetic credit card transaction logs paired with generated merchant receipt image samples containing intentional fraud perturbations.
Challenge setup
Evaluates accuracy of visual text extraction and fraud classification decision accuracy.
Challenge input
JSON containing credit_card_log and base64_image string
Google ADK
Google Agent Development Kit for building multimodal agents with Gemini.
Evaluated output
JSON containing alert_level, amount_mismatch, and rationale
- Verifies that amount discrepancies are correctly identified
- Accuracy score on test set containing mixed valid and fraudulent alerts • target: 94 • range: 0-100
- Benchmark: FinMultimodal-Fraud-Bench
- Fraud Triage Accuracy target: 94
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Google · Required
- Google ADK · Optional
- Google Jules · Optional
- Google Jules · Optional
Evaluation contract
- mismatch_detection · Weight 1
- fraud_triage_accuracy · Weight 1
Metrics
- fraud_triage_accuracy · Target 94
Benchmarks
- FinMultimodal-Fraud-Bench
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing credit_card_log and base64_image string
Output: JSON containing alert_level, amount_mismatch, and rationale
Execution harness: Python
Evaluation tools
- Google ADK · Agent Framework
Recipe freeze marker
No recipe freeze time is recorded.