Skip to main content
Loading navigation
Back to challenges

Multimodal Anti-Scam Triage System with Claude Agents SDK and Claude 4.1 Opus

Develop a high-precision anti-scam triage agent using Claude Agents SDK and Claude 4.1 Opus to analyze suspicious SMS, voice transcripts, and phishing links. Identify zero-day social engineering vectors and generate rapid blocklist updates for Smart Nation defenses.

Workflow AutomationHosted by Vera
Sign in free to start

What you are building

Construct an advanced anti-scam agent using Claude Agents SDK and Claude 4.1 Opus to analyze social engineering tactics and trigger immediate threat mitigations.

How work is evaluated

Evaluation

Evaluates scam classification accuracy, threat indicator extraction, and policy enforcement speed.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

scam_detected

Verifies that known phishing messages are classified as scam

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

false_positive_rate

Rate of legitimate messages incorrectly flagged as scams • target: 0.005 • range: 0-0.02

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Master Anthropic's Claude Agents SDK for state management and complex tool execution

  • Leverage Claude 4.1 Opus extended reasoning capabilities to uncover deep multi-stage scam tactics

  • Build multimodal scam feature extractors analyzing text, domain registration patterns, and payment links

  • Automate public sector anti-scam escalation actions with verifiable chain-of-thought rationale

Resources

Dataset notes

Dataset containing 100 labeled Singapore SMS and messaging app fraud reports.

Challenge setup

Evaluates scam classification accuracy, threat indicator extraction, and policy enforcement speed.

Challenge input

JSON object with message_body, sender_id, and attached_urls

Claude Agents SDK

Provides structured tool use and agent loops with Claude models.

Claude 4.1 Opus

Delivers extended thinking capabilities for intricate threat evaluation.

Evaluated output

JSON object with classification (scam/legitimate), risk_level, extracted_indicators, and mitigation_action

Evaluation checks
  • Verifies that known phishing messages are classified as scam
  • Rate of legitimate messages incorrectly flagged as scams • target: 0.005 • range: 0-0.02
Configured evaluation evidence
  • Benchmark: PhishBench Cyber Threat Evaluation
  • False Positive Rate target: 0.5%
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Environment
  • Modal · Optional
Action Space
  • Agentic Dev · Optional
Policy Serving
  • Claude 4.1 Opus · Required
  • Modal · Optional

Evaluation contract

  • scam_detected · Weight 1
  • false_positive_rate · Weight 1

Metrics

  • false_positive_rate · Target 0.005

Benchmarks

  • PhishBench Cyber Threat Evaluation
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON object with message_body, sender_id, and attached_urls

    Output: JSON object with classification (scam/legitimate), risk_level, extracted_indicators, and mitigation_action

    Execution harness: Python

Evaluation tools

  • Claude Agents SDK · Agent Framework
  • Claude 4.1 Opus · Reasoning LLM

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Multimodal Anti-Scam Triage System with Claude Agents SDK and Claude 4.1 Opus