Skip to main content
Loading navigation
Back to challenges

Image Integrity Verification Agent with Mastra AI and Llama 3.3 70B

Scientific image manipulation in antibody validation compromises research integrity and regulatory compliance. Developers will construct a Mastra AI workflow leveraging Llama 3.3 70B to inspect figure panels, detect duplication or forensic tampering in antibody validation data, and generate verifiable integrity audit scores.

Data ScienceHosted by Vera
Sign in free to start

What you are building

Build a Mastra AI forensic agent using Llama 3.3 70B to analyze publication figure images and report manipulation anomalies.

How work is evaluated

Evaluation

Evaluates forensic image manipulation detection rate, false positive balance, and Mastra state execution integrity.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

forensic_flag_check

Ensures correctly manipulated images are flagged.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

detection_f1_score

F1 score of image manipulation anomaly identification. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Configure a Mastra AI workflow with state management for image inspection.

  • Integrate Llama 3.3 70B multimodal endpoints to analyze western blot figure panels.

  • Detect pixel duplication, splicing, and brightness manipulation anomalies.

  • Generate structured image integrity forensic audits with bounding-box metadata.

Resources

Dataset notes

Curated set of biological antibody validation figures containing benchmarked synthetic and real-world image manipulation samples.

Challenge setup

Evaluates forensic image manipulation detection rate, false positive balance, and Mastra state execution integrity.

Challenge input

JSON containing image URL and metadata.

Mastra AI

Provides structured workflow states and agent integration for JS/TS environments.

Llama 3.3 70B

High-capacity open weights model for complex visual analysis.

Evaluated output

JSON containing manipulation detected boolean, confidence, and anomaly list.

Evaluation checks
  • Ensures correctly manipulated images are flagged.
  • F1 score of image manipulation anomaly identification. • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: Scientific Fraud Detection Benchmark
  • Detection F1 Score target: 0.9
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Llama 3.3 70B · Required
  • Mastra AI · Optional
  • Featuretools · Optional
Orchestration
  • Mastra AI · Optional

Evaluation contract

  • forensic_flag_check · Weight 1
  • detection_f1_score · Weight 1

Metrics

  • detection_f1_score · Target 0.9

Benchmarks

  • Scientific Fraud Detection Benchmark
  • biotech

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing image URL and metadata.

    Output: JSON containing manipulation detected boolean, confidence, and anomaly list.

    Execution harness: JavaScript

Evaluation tools

  • Mastra AI · TypeScript Workflow Engine
  • Llama 3.3 70B · Multimodal Vision & Language Model

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Image Integrity Verification Agent with Mastra AI and Llama 3.3 70B