Back to challenges
Challenge

Image Integrity Verification Agent with Mastra AI and Llama 3.3 70B

Scientific image manipulation in antibody validation compromises research integrity and regulatory compliance. Developers will construct a Mastra AI workflow leveraging Llama 3.3 70B to inspect figure panels, detect duplication or forensic tampering in antibody validation data, and generate verifiable integrity audit scores.

Data ScienceHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a Mastra AI forensic agent using Llama 3.3 70B to analyze publication figure images and report manipulation anomalies.

Delivery guide

How work is evaluated

Evaluation

Evaluates forensic image manipulation detection rate, false positive balance, and Mastra state execution integrity.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Loading datasets...
Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

forensic_flag_check

Ensures correctly manipulated images are flagged.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

detection_f1_score

F1 score of image manipulation anomaly identification. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Configure a Mastra AI workflow with state management for image inspection.

  • Integrate Llama 3.3 70B multimodal endpoints to analyze western blot figure panels.

  • Detect pixel duplication, splicing, and brightness manipulation anomalies.

  • Generate structured image integrity forensic audits with bounding-box metadata.

Resources and assets

Reference links and supporting material

Dataset notes

Curated set of biological antibody validation figures containing benchmarked synthetic and real-world image manipulation samples.

How this agent runs

Evaluates forensic image manipulation detection rate, false positive balance, and Mastra state execution integrity.

Challenge input

JSON containing image URL and metadata.

Mastra AI

Provides structured workflow states and agent integration for JS/TS environments.

Llama 3.3 70B

High-capacity open weights model for complex visual analysis.

Evaluated output

JSON containing manipulation detected boolean, confidence, and anomaly list.

Checks for
  • Ensures correctly manipulated images are flagged.
  • F1 score of image manipulation anomaly identification. • target: 0.9 • range: 0-1
Proof of success
  • Benchmark: Scientific Fraud Detection Benchmark
  • Detection F1 Score target: 0.9
  • 1 public reference case
Runtime evidence
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Llama 3.3 70B · Required
  • Mastra AI · Optional
  • Featuretools · Optional
Orchestration
  • Mastra AI · Optional

Evaluation contract

  • forensic_flag_check · Weight 1
  • detection_f1_score · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Image Integrity Verification Agent with Mastra AI and Llama 3.3 70B