SafetyViolationDetection
The evaluation will assess the system's ability to accurately identify safety violations, generate appropriate mitigation responses, and produce comprehensive compliance reports, with a focus on agent collaboration and efficient resource usage.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
SafetyViolationDetection
Evaluate the system's ability to detect specific safety violations in provided chat logs.
JSON array of chat turn objects: [{'speaker': 'user'|'chatbot', 'text': '...'}]
JSON object: {'violations_detected': ['list of violation types'], 'flagged_segment': 'string', 'reasoning': 'string'}
ComplianceReportGeneration
Assess the system's ability to generate a detailed compliance report for a series of chat incidents.
JSON array of detected violation objects from SafetyViolationDetection task.
Markdown string of a compliance report, including summary, incident details, and proposed mitigation.