Build OpenAI Agents SDK Multilingual DPI Voice Agent with Cartesia TTS
Non-English speaking citizens face accessibility barriers when navigating digital public services like Aadhaar and PM-Kisan portal updates. Build an interactive voice assistance agent using the OpenAI Agents SDK and Cartesia streaming TTS to provide low-latency audio support with under 500ms response latency.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop a real-time voice assistance agent leveraging OpenAI Agents SDK function calling and Cartesia audio synthesis for regional language assistance.
How work is evaluated
Evaluates response accuracy and audio response generation latency.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
latency_check
Verifies latency is below 500ms threshold
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
latency_ms
End-to-end response generation latency in milliseconds • target: 450 • range: 0-2000
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Implement OpenAI Agents SDK runner loops with streaming audio integration
Connect Cartesia low-latency TTS for ultra-fast text-to-speech rendering in Hindi and Tamil
Build dynamic schema tools to query citizen service status endpoints
Ensure voice latency stays under 500ms end-to-end
Reference links and supporting material
Synthetic citizen service dataset containing portal update histories and localized voice queries.
How this agent runs
Evaluates response accuracy and audio response generation latency.
Challenge input
JSON audio transcript and target language
OpenAI Agents SDK
Agent control loops and function execution
Cartesia
Ultra low latency streaming voice engine
Evaluated output
JSON object with status, answer, and latency ms
- Verifies latency is below 500ms threshold
- End-to-end response generation latency in milliseconds • target: 450 • range: 0-2000
- Benchmark: Multilingual Voice Agent Benchmark
- Latency Ms target: 450
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
View technical recipe
Configured tools
- Cartesia · Required
- OpenAI · Optional
- Alibi Detect · Optional
Evaluation contract
- latency_check · Weight 1
- latency_ms · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.