Back to evaluations
Public evaluation
substation_query_task
Evaluates accuracy of answers extracted from substation contract PDFs and latency over LiveKit voice channel.
Evaluation type
task based
Challenge
Build a Voice-Enabled Substation Procurement RAG Agent with LlamaIndex and LiveKit
Difficulty
Advanced
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
substation_query_task
Query specific transformer ratings and breaker isolation protocols from contract index.
Input format
Audio chunk or audio transcript string query
Output format
JSON containing string response, source_node_id, latency_ms