Back to evaluations
Public evaluation
Vulnerability Analysis Accuracy
The evaluation focuses on the accuracy of the LLM parsing, the reliability of the Temporal workflow, and the structural integrity of the generated STIX intelligence.
Evaluation type
task based
Challenge
Automated Zero-Day Triage Pipeline
Difficulty
Advanced
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Vulnerability Analysis Accuracy
Parses a raw SonicWall advisory and extracts CVE IDs, affected versions, and severity.
Input format
Raw text blob (security advisory)
Output format
JSON object with keys: cve_id, affected_versions, severity_score, mitigations