Back to evaluations
Public evaluation
vasp_audit_eval
Evaluates schema strictness and compliance rule checking against VASP submission documents.
Evaluation type
task based
Challenge
Implement a VASP License Risk Auditor with Pydantic AI & AI SDK
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
vasp_audit_eval
Validates compliance score and entity validation output for a VASP applicant.
Input format
JSON containing entity details and custody architecture
Output format
JSON conforming to strict Pydantic AuditResult model