Back to challenges
Challenge

Automated Image Misconduct Detection RAG Pipeline with LlamaIndex

Automated image duplication and manipulation in biomedical publications compromise scientific integrity. Build a LlamaIndex-powered audit pipeline using MutableAI code generation to detect duplicated gel bands and image manipulations, outputting structured provenance reports with actionable verification scores.

Data ScienceHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Construct an indexed figure extraction pipeline with LlamaIndex to query biomedical image metadata and flag image duplication anomalies.

Delivery guide

How work is evaluated

Evaluation

Evaluates candidate capability to index figures, detect manipulated panels, and output valid structured audit summaries.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

JSON Audit Schema Validation

Verifies output conforms strictly to the validation schema.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Detection Accuracy

F1-score of correctly flagged manipulated image panels. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Index publication figure metadata and image feature embeddings using LlamaIndex.

  • Integrate MutableAI agentic refactoring tools to optimize computer vision parsing routines.

  • Detect image duplication, contrast manipulation, and splicing in Western blots.

  • Generate structured JSON verification audits adhering to publication integrity standards.

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic and annotated biomedical figure panel dataset containing clean Western blots, cropped duplicates, and contrast-adjusted microscopy images.

How this agent runs

Evaluates candidate capability to index figures, detect manipulated panels, and output valid structured audit summaries.

Challenge input

JSON containing figure manifest with image file paths

LlamaIndex

Provides structural indexing for multi-modal publication figures.

MutableAI

Refactors feature extraction logic for high-performance execution.

Evaluated output

JSON containing detected duplicate regions, similarity scores, and flag boolean

Checks for
  • Verifies output conforms strictly to the validation schema.
  • F1-score of correctly flagged manipulated image panels. • target: 0.9 • range: 0-1
Proof of success
  • Benchmark: BioImageIntegrity-Bench
  • Detection Accuracy target: 0.9
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Llama Index · Required
  • MutableAI · Optional
  • Cohere · Optional

Evaluation contract

  • JSON Audit Schema Validation · Weight 1
  • Detection Accuracy · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Automated Image Misconduct Detection RAG Pipeline with LlamaIndex