Public Health Misinformation Triage Agent with LangChain and Instructor
Public health communications require structured extraction of scientific claims and empathetic counter-messaging. You will build a LangChain agent using Instructor to validate public health claims against medical literature, outputting structured JSON with empathy scores and accuracy metrics achieving 90% validation accuracy.
What you are building
Implement a LangChain graph workflow using Instructor for structured output validation against scientific literature databases.
How work is evaluated
Evaluates structured JSON extraction accuracy and claim validation performance against gold standard public health claims.
Datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
valid_json_schema
Output must match expected JSON schema exactly
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
validation_accuracy
Accuracy of claim factual classification • target: 0.9 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Learning goals
Build structured LLM outputs using Instructor and Pydantic schema validation
Construct multi-step claim verification chains using LangChain and LangGraph
Evaluate empathetic messaging tone alongside scientific factual accuracy
Resources
Dataset of 500 vaccine-related public health claims with annotated factual correctness and empathy ratings.
Challenge setup
Evaluates structured JSON extraction accuracy and claim validation performance against gold standard public health claims.
Challenge input
JSON object with claim_text string
LangChain
Required framework for building LLM stateful chains.
Instructor
Enforces strict Pydantic JSON outputs from LLM responses.
Evaluated output
JSON object with is_factual boolean, confidence float, and citations array
- Output must match expected JSON schema exactly
- Accuracy of claim factual classification • target: 0.9 • range: 0-1
- Benchmark: HealthFact-Eval
- Validation Accuracy target: 0.9
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
- Instructor · Required
- Langchain · Optional
- Pangu 3.0 · Optional
- Langchain · Optional
Evaluation contract
- valid_json_schema · Weight 1
- validation_accuracy · Weight 1
Metrics
- validation_accuracy · Target 0.9
Benchmarks
- HealthFact-Eval
- biotech
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON object with claim_text string
Output: JSON object with is_factual boolean, confidence float, and citations array
Execution harness: Python
Evaluation tools
- LangChain · Agent Orchestration Framework
- Instructor · Structured Extraction Library
Recipe freeze marker
No recipe freeze time is recorded.