Public evaluation

CopyrightAnalysisTask

Evaluation will assess the agent system's accuracy in identifying content provenance, its ability to apply copyright principles, and the depth of its analysis for potential infringements, alongside the effectiveness of RAG and tool integration.

Evaluation type

task based

Challenge

Multi-Agent Content Rights Auditor

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

CopyrightAnalysisTask

Analyze a piece of generated text against a simulated internal content database and legal corpus to determine its originality and potential copyright issues.

Input format

{'generated_text_id': 'string', 'generated_content': 'string', 'author_name': 'string'}

Output format

{'originality_score': 'float', 'identified_sources': [{'source_id': 'string', 'match_percentage': 'float', 'source_type': 'internal|external'}], 'potential_infringement_risk': 'low|medium|high', 'risk_explanation': 'string'}

Task 2

ProvenanceResearchTask

Trace the potential origin of a piece of content by searching simulated internal databases and external web sources via MCP, and provide a provenance report.

Input format

{'content_fragment': 'string'}

Output format

{'provenance_status': 'found|unclear|not_found', 'top_matches': [{'url': 'string', 'match_score': 'float', 'source_type': 'internal|external', 'snippet': 'string'}], 'search_duration_ms': 'int'}