Skip to main content
Back to challenges
Challenge

Newsroom Fact-Check & Bias Detector

Inspired by the challenges and opportunities of AI in newsrooms, this challenge focuses on building a sophisticated agentic system for automated fact-checking and bias detection. You will use LangGraph to design a dynamic, graph-based workflow for agents to iteratively analyze news articles, cross-reference information, and identify potential biases. Claude Opus 4.1, with its long context window and nuanced reasoning, will be central to understanding complex narratives and detecting subtle biases, while a secondary model (e.g., Grok-2) could provide real-time trending context.

NLPHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop a LangGraph-based multi-agent system using Claude Opus 4.1 for nuanced news fact-checking and bias detection with iterative workflows.

Delivery guide

How work is evaluated

Evaluation

The system will be evaluated on its ability to accurately identify factual errors, detect subtle biases, and provide supporting evidence for its conclusions on a given news article. Efficiency and robustness of the LangGraph workflow will also be assessed.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Learning goals

What you should walk away with

  • Integrate multiple information sources (RAG, web search) for evidence gathering.

  • Develop metrics and methods for scoring factual accuracy and bias levels in news content.

  • Leverage Claude Opus 4.1 for deep contextual understanding and bias identification.

  • Design and implement a graph-based agent workflow using LangGraph for iterative fact-checking.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

The system will be evaluated on its ability to accurately identify factual errors, detect subtle biases, and provide supporting evidence for its conclusions on a given news article. Efficiency and robustness of the La...

Challenge input

{ "article_text": "string", "context_keywords": ["string"] }

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{ "fact_check_report": [ { "claim": "string", "status": "accurate|inaccurate|unverified", "evidence": ["string"] } ], "bias_assessment": { "overall...

Evaluation checks
  • Are all significant factual claims in the article identified and assessed?
  • Is each 'inaccurate' or 'unverified' claim supported by credible evidence?
Configured evaluation evidence
  • Factual Accuracy Score target: 90
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • Are all significant factual claims in the article identified and assessed?
  • Is each 'inaccurate' or 'unverified' claim supported by credible evidence?

Metrics

  • Factual Accuracy Score · Target 90
  • Bias Detection Precision · Target 80

Workflow tasks

  1. Task 1

    Input: { "article_text": "string", "context_keywords": ["string"] }

    Output: { "fact_check_report": [ { "claim": "string", "status": "accurate|inaccurate|unverified", "evidence": ["string"] } ], "bias_assessment": { "overall_bias": "neutral|liberal|conservative|sensationalist", "bias_points": [ { "type": "string", "location": "string", "explanation": "string" } ] }, "confidence_score": "float" }

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Newsroom Fact-Check & Bias Detector