NavigationQuery
The evaluation will assess the agent's ability to provide accurate, context-aware, and timely navigational and safety assistance via a hands-free voice interface. Focus will be on conversational flow, multimodal understanding, and tool integration correctness.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
NavigationQuery
Evaluates the agent's ability to provide directions and answer navigation-related questions based on simulated real-time location.
{ "user_utterance": "string", "current_location": { "latitude": "float", "longitude": "float" }, "current_activity": "string" }
{ "agent_response": "string", "api_calls_made": "array" }
SafetyAlertResponse
Tests the agent's ability to detect potential hazards (simulated) and provide appropriate safety warnings or actions.
{ "user_utterance": "string", "current_location": { "latitude": "float", "longitude": "float" }, "detected_hazard": "string", "current_activity": "string" }
{ "agent_response": "string", "safety_action_recommended": "string" }