Multimodal Anti-Scam Triage System with Claude Agents SDK and Claude 4.1 Opus
Develop a high-precision anti-scam triage agent using Claude Agents SDK and Claude 4.1 Opus to analyze suspicious SMS, voice transcripts, and phishing links. Identify zero-day social engineering vectors and generate rapid blocklist updates for Smart Nation defenses.
What you are building
The core problem, expected build, and operating context for this challenge.
Construct an advanced anti-scam agent using Claude Agents SDK and Claude 4.1 Opus to analyze social engineering tactics and trigger immediate threat mitigations.
How work is evaluated
Evaluates scam classification accuracy, threat indicator extraction, and policy enforcement speed.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
scam_detected
Verifies that known phishing messages are classified as scam
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
false_positive_rate
Rate of legitimate messages incorrectly flagged as scams • target: 0.005 • range: 0-0.02
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master Anthropic's Claude Agents SDK for state management and complex tool execution
Leverage Claude 4.1 Opus extended reasoning capabilities to uncover deep multi-stage scam tactics
Build multimodal scam feature extractors analyzing text, domain registration patterns, and payment links
Automate public sector anti-scam escalation actions with verifiable chain-of-thought rationale
Reference links and supporting material
Dataset containing 100 labeled Singapore SMS and messaging app fraud reports.
Challenge setup
Evaluates scam classification accuracy, threat indicator extraction, and policy enforcement speed.
Challenge input
JSON object with message_body, sender_id, and attached_urls
Claude Agents SDK
Provides structured tool use and agent loops with Claude models.
Claude 4.1 Opus
Delivers extended thinking capabilities for intricate threat evaluation.
Evaluated output
JSON object with classification (scam/legitimate), risk_level, extracted_indicators, and mitigation_action
- Verifies that known phishing messages are classified as scam
- Rate of legitimate messages incorrectly flagged as scams • target: 0.005 • range: 0-0.02
- Benchmark: PhishBench Cyber Threat Evaluation
- False Positive Rate target: 0.5%
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Modal · Optional
- Agentic Dev · Optional
- Claude 4.1 Opus · Required
- Modal · Optional
Evaluation contract
- scam_detected · Weight 1
- false_positive_rate · Weight 1
Metrics
- false_positive_rate · Target 0.005
Benchmarks
- PhishBench Cyber Threat Evaluation
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON object with message_body, sender_id, and attached_urls
Output: JSON object with classification (scam/legitimate), risk_level, extracted_indicators, and mitigation_action
Execution harness: Python
Evaluation tools
- Claude Agents SDK · Agent Framework
- Claude 4.1 Opus · Reasoning LLM
Recipe freeze marker
No recipe freeze time is recorded.