Public evaluation

Diagnostic Accuracy

The system must accurately identify the 'withdrawal phase' findings across 5 different colonoscopy reports, correctly linking visual polyp images to the textual size description.

Evaluation type

task based

Challenge

Multimodal Macro-Reasoning RAG for Colonoscopy Analysis with LlamaIndex and Together AI

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

Diagnostic Accuracy

Tests if the RAG system correctly associates images with the correct textual clinical findings.

Input format

Zip file containing 10 images and 5 report PDFs.

Output format

Structured Diagnostic Summary (JSON).

Linked challenge

Multimodal Macro-Reasoning RAG for Colonoscopy Analysis with LlamaIndex and Together AI

Standard RAG systems often fail on complex technical documents like medical reports because they focus on 'micro-matching' (finding specific text chunks) while ignoring 'macro-reasoning' (understanding the overall procedural context). Inspired by the 'DualG-MRAG' and colonoscopy foundation model research, this challenge requires you to build a multimodal RAG pipeline using LlamaIndex. You will implement a dual-graph approach where one graph tracks the high-level medical procedure (macro) and the other handles specific visual/textual findings (micro). You will use Together AI's high-speed inference for Llama 3.2 Vision to process both medical imagery and textual routine reports.

Open challenge