Automated Image Misconduct Detection RAG Pipeline with LlamaIndex
Automated image duplication and manipulation in biomedical publications compromise scientific integrity. Build a LlamaIndex-powered audit pipeline using MutableAI code generation to detect duplicated gel bands and image manipulations, outputting structured provenance reports with actionable verification scores.
What you are building
The core problem, expected build, and operating context for this challenge.
Construct an indexed figure extraction pipeline with LlamaIndex to query biomedical image metadata and flag image duplication anomalies.
How work is evaluated
Evaluates candidate capability to index figures, detect manipulated panels, and output valid structured audit summaries.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
JSON Audit Schema Validation
Verifies output conforms strictly to the validation schema.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Detection Accuracy
F1-score of correctly flagged manipulated image panels. • target: 0.9 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Index publication figure metadata and image feature embeddings using LlamaIndex.
Integrate MutableAI agentic refactoring tools to optimize computer vision parsing routines.
Detect image duplication, contrast manipulation, and splicing in Western blots.
Generate structured JSON verification audits adhering to publication integrity standards.
Reference links and supporting material
Synthetic and annotated biomedical figure panel dataset containing clean Western blots, cropped duplicates, and contrast-adjusted microscopy images.
Challenge setup
Evaluates candidate capability to index figures, detect manipulated panels, and output valid structured audit summaries.
Challenge input
JSON containing figure manifest with image file paths
LlamaIndex
Provides structural indexing for multi-modal publication figures.
MutableAI
Refactors feature extraction logic for high-performance execution.
Evaluated output
JSON containing detected duplicate regions, similarity scores, and flag boolean
- Verifies output conforms strictly to the validation schema.
- F1-score of correctly flagged manipulated image panels. • target: 0.9 • range: 0-1
- Benchmark: BioImageIntegrity-Bench
- Detection Accuracy target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- Llama Index · Required
- MutableAI · Optional
- Cohere · Optional
Evaluation contract
- JSON Audit Schema Validation · Weight 1
- Detection Accuracy · Weight 1
Metrics
- Detection Accuracy · Target 0.9
Benchmarks
- BioImageIntegrity-Bench
- biotech
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing figure manifest with image file paths
Output: JSON containing detected duplicate regions, similarity scores, and flag boolean
Execution harness: Python
Evaluation tools
- LlamaIndex · RAG Indexing
- MutableAI · AI Code Optimization
Recipe freeze marker
No recipe freeze time is recorded.