IdentityRiskAssessment
The agent will be evaluated on its ability to accurately identify pre-defined identity-related security risks within a given knowledge graph, provide clear explanations for the risks, and suggest appropriate mitigation strategies. Evaluation will also cover the efficiency of its graph traversal and reasoning process.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
IdentityRiskAssessment
Given a knowledge graph representing enterprise identities and access, identify specific security vulnerabilities (e.g., dormant accounts with high privileges, unauthorized access paths).
{'knowledge_graph_data': {'nodes': [{'id': 'user_alice', 'type': 'user', 'roles': ['admin']}, {'id': 'server_db', 'type': 'system', 'security_level': 'confidential'}], 'edges': [{'source': 'user_alice', 'target': 'server_db', 'relation': 'accesses'}], 'query': 'Identify dormant high-privilege accounts.'}}
{'identified_risks': [{'risk_type': 'DormantAdmin', 'description': 'User Bob has admin privileges but no recent activity.', 'mitigation': 'Revoke privileges or disable account'}], 'reasoning_path': ['...']}