Proactive Voice Safety Assistant
This challenge focuses on developing a modern, proactive voice-enabled AI assistant designed to provide safety and well-being checks for individuals living alone. Unlike a simple button press, your solution should leverage advanced conversational AI, real-time voice synthesis/recognition, and agentic workflows to personalize interactions and proactively detect potential issues. The assistant must maintain long-term memory of user preferences and integrate with external alert systems, all while prioritizing user privacy and a natural conversational experience.
What you are building
Develop a real-time, proactive voice AI assistant using Bland AI and Claude Sonnet for personalized safety check-ins with long-term memory.
How work is evaluated
The assistant will be evaluated on its ability to conduct natural, personalized conversations, successfully perform scheduled check-ins, detect predefined 'distress' indicators in speech/text, and accurately trigger alerts while leveraging its long-term memory. Latency of voice interaction will also be a key factor.
Datasets
Public datasets, plus the private datasets you have access to.
Learning goals
Master real-time voice AI integration using Bland AI for seamless, natural language interactions with low latency.
Implement sophisticated conversational logic and personalized responses with Claude Sonnet, managing complex dialogue states and intent recognition.
Design proactive agentic workflows using Mindset AI to schedule check-ins, detect anomalies from conversation patterns, and trigger configurable alerts.
Integrate WebAI to allow the voice assistant to interact with web services (e.g., weather updates, news summaries, emergency contact services) based on user requests or proactive needs.
Build a persistent, personalized long-term memory using ChromaDB to store user preferences, conversation history, and key safety information, enabling highly contextual and empathetic interactions.
Resources
Sample data for 2 tasks
Challenge setup
The assistant will be evaluated on its ability to conduct natural, personalized conversations, successfully perform scheduled check-ins, detect predefined 'distress' indicators in speech/text, and accurately trigger a...
Challenge input
{'user_id': 'string', 'initial_prompt': 'string', 'memory_state': {'last_interaction_topic': 'string', 'preferred_greeting': 'string'}}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'transcript': [{'speaker': 'user|assistant', 'text': 'string'}], 'alert_triggered': 'boolean'}
- Ensures the average turn-taking latency (user speech end to assistant speech start) is below 1 second.
- Verifies that alerts are triggered correctly for distress scenarios and not for normal interactions.
- PersonalizationScore target: 0.8
- Python execution harness
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Ensures the average turn-taking latency (user speech end to assistant speech start) is below 1 second.
- Verifies that alerts are triggered correctly for distress scenarios and not for normal interactions.
Metrics
- PersonalizationScore · Target 0.8
- ConversationalFluency · Target 0.85
Workflow tasks
- Task 1
Input: {'user_id': 'string', 'initial_prompt': 'string', 'memory_state': {'last_interaction_topic': 'string', 'preferred_greeting': 'string'}}
Output: {'transcript': [{'speaker': 'user|assistant', 'text': 'string'}], 'alert_triggered': 'boolean'}
Execution harness: Python
- Task 2
Input: {'user_id': 'string', 'conversation_transcript': [{'speaker': 'user|assistant', 'text': 'string'}]}
Output: {'alert_triggered': 'boolean', 'reason': 'string'}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.