Back to evaluations
Public evaluation

DSAComplianceReportGeneration

The evaluation will assess the system's ability to accurately identify potential DSA compliance issues from simulated platform data, leveraging advanced RAG and ethical reasoning to generate a justified compliance report.

Evaluation type
task based
Challenge
DSA Compliance Monitoring
Difficulty
Advanced
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
1
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

DSAComplianceReportGeneration

Evaluates the system's capacity to process mock platform data (posts, user interactions), cross-reference with DSA regulations using RAG, identify potential violations, and generate a structured compliance report with justifications and evidence.

Input format

{ 'dsa_articles_to_monitor': ['string'], 'mock_platform_data': [{ 'post_id': 'string', 'author': 'string', 'content': 'string', 'timestamp': 'string', 'visibility': 'public|private', 'research_access_policy': 'string' }], 'mock_dsa_legal_text': 'string' }

Output format

{ 'compliance_report': { 'platform_id': 'string', 'compliance_status': 'compliant|non-compliant|partial', 'violations_found': [{ 'article_id': 'string', 'description': 'string', 'evidence_ids': ['string'], 'justification': 'string' }], 'recommendations': ['string'] }, 'agent_communication_log': [{ 'sender': 'string', 'receiver': 'string', 'protocol': 'A2A', 'message': 'string', 'timestamp': 'string' }] }