Skip to main content
Back to challenges
Challenge

Multi-Agent Content Rights Auditor

This challenge involves building an agentic system for automated content rights auditing. Participants will design a multi-agent team using Langroid, where agents specialize in legal text analysis, content provenance research, and copyright infringement detection. Claude Opus 4.5, known for its advanced reasoning and nuanced text understanding, will power the legal and analytical agents. The system will leverage LlamaIndex for advanced RAG across simulated legal and content databases, and integrate web crawling tools (e.g., Firecrawl via MCP) to assess potential external infringements.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an automated content rights auditing system with Langroid agents, Claude Opus 4.5 for legal analysis, LlamaIndex for RAG, and MCP for tool integration.

Delivery guide

How work is evaluated

Evaluation

Evaluation will assess the agent system's accuracy in identifying content provenance, its ability to apply copyright principles, and the depth of its analysis for potential infringements, alongside the effectiveness of RAG and tool integration.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Learning goals

What you should walk away with

  • Implement advanced RAG using LlamaIndex for legal and content repositories.

  • Integrate web crawling and external data sources via MCP for broader context.

  • Design and orchestrate a role-based multi-agent system for content auditing.

  • Utilize Claude Opus 4.1 for highly nuanced legal and textual analysis.

  • Generate detailed reports on content rights and potential infringements.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 2 tasks

Challenge setup

Evaluation will assess the agent system's accuracy in identifying content provenance, its ability to apply copyright principles, and the depth of its analysis for potential infringements, alongside the effectiveness o...

Challenge input

{'generated_text_id': 'string', 'generated_content': 'string', 'author_name': 'string'}

Agent execution

No agent tool is listed for this challenge.

Evaluated output

{'originality_score': 'float', 'identified_sources': [{'source_id': 'string', 'match_percentage': 'float', 'source_type': 'internal|external'}], 'p...

Evaluation checks
  • Verify that distinct Langroid agents are instantiated and interact correctly as per their roles.
  • Confirm that LlamaIndex successfully retrieves relevant context from both legal and content databases to in...
  • Ensure the MCP integration successfully triggers a simulated web crawl (e.g., using Firecrawl) and retrieve...
Configured evaluation evidence
  • CopyrightRiskAccuracy target: 0.85
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

No tool records are attached.

Evaluation contract

  • Verify that distinct Langroid agents are instantiated and interact correctly as per their roles.
  • Confirm that LlamaIndex successfully retrieves relevant context from both legal and content databases to inform agent decisions.
  • Ensure the MCP integration successfully triggers a simulated web crawl (e.g., using Firecrawl) and retrieves external content for analysis.

Metrics

  • CopyrightRiskAccuracy · Target 0.85
  • ProvenanceMatchRelevance · Target 0.9

Workflow tasks

  1. Task 1

    Input: {'generated_text_id': 'string', 'generated_content': 'string', 'author_name': 'string'}

    Output: {'originality_score': 'float', 'identified_sources': [{'source_id': 'string', 'match_percentage': 'float', 'source_type': 'internal|external'}], 'potential_infringement_risk': 'low|medium|high', 'risk_explanation': 'string'}

    Execution harness: Python

  2. Task 2

    Input: {'content_fragment': 'string'}

    Output: {'provenance_status': 'found|unclear|not_found', 'top_matches': [{'url': 'string', 'match_score': 'float', 'source_type': 'internal|external', 'snippet': 'string'}], 'search_duration_ms': 'int'}

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Multi-Agent Content Rights Auditor