AI-Assisted Flight Operations Agent
This challenge focuses on building an AI-powered flight assistant using the Vercel AI SDK. This assistant will provide real-time guidance, perform safety checks, and assist with complex operational procedures through a conversational, voice-enabled interface. You will integrate Claude Sonnet 4 for robust reasoning and Llama 3 for quick, localized responses (via Hugging Face Inference Endpoints). The system will leverage a low-code automation platform like Hyperbolic for connecting to simulated flight control systems and OpenTelemetry for robust observability of AI interactions and system state, ensuring safety and compliance in critical aerospace operations.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an AI-powered, voice-enabled flight assistant using Vercel AI SDK, Claude Sonnet 4, and Llama 3 for real-time operational guidance and safety checks.
How work is evaluated
The evaluation will assess the assistant's ability to accurately understand voice commands, execute correct flight procedures via tools, provide timely and relevant information, and maintain a high level of system observability.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
CorrectToolExecution
The assistant must execute the correct sequence of tools based on the voice command and flight state.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
ContextualResponseAccuracy
The 'agent_response' must be contextually appropriate and informative based on the simulated actions.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
ResponseLatency
Time taken from voice command processing to agent response, lower is better. • target: 300 • range: 0-1000
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
ObservabilityCompleteness
Score based on the completeness and correctness of OpenTelemetry traces for the interaction, higher is better. • target: 90 • range: 0-100
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master the Vercel AI SDK for building streaming, tool-using conversational interfaces in TypeScript/JavaScript.
Implement voice input and output using Fixie (or a similar Web Speech API integration) to create a natural, hands-free interaction experience.
Design intelligent tool calls within the AI SDK for interacting with a simulated flight control system (e.g., 'check_fuel_level', 'initiate_autopilot_sequence').
Integrate Claude Sonnet 4 for critical reasoning tasks and complex procedure interpretation, leveraging its strong safety and reliability features.
Utilize Llama 3 via Hugging Face Inference Endpoints for quicker, context-specific responses or simpler control commands.
Connect the AI SDK application to a simulated flight system via Hyperbolic (or a mock API gateway) for triggering external actions and fetching real-time data.
Implement OpenTelemetry tracing and logging within the AI SDK application to monitor user interactions, agent decisions, and tool executions for auditing and debugging flight-critical operations.
Reference links and supporting material
Sample data for 1 tasks
Challenge setup
The evaluation will assess the assistant's ability to accurately understand voice commands, execute correct flight procedures via tools, provide timely and relevant information, and maintain a high level of system obs...
Challenge input
{'voice_command_transcript': 'string', 'simulated_flight_state': 'dict', 'expected_tool_calls': 'list_of_dicts'}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'agent_response': 'string', 'executed_tool_calls': 'list_of_dicts', 'open_telemetry_trace_id': 'string'}
- The assistant must execute the correct sequence of tools based on the voice command and flight state.
- The 'agent_response' must be contextually appropriate and informative based on the simulated actions.
- Time taken from voice command processing to agent response, lower is better. • target: 300 • range: 0-1000
- ResponseLatency target: 300
- 1 public reference case
- JavaScript execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- CorrectToolExecution · Weight 1
- ContextualResponseAccuracy · Weight 1
- ResponseLatency · Weight 1
- ObservabilityCompleteness · Weight 1
Metrics
- ResponseLatency · Target 300
- ObservabilityCompleteness · Target 90
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: {'voice_command_transcript': 'string', 'simulated_flight_state': 'dict', 'expected_tool_calls': 'list_of_dicts'}
Output: {'agent_response': 'string', 'executed_tool_calls': 'list_of_dicts', 'open_telemetry_trace_id': 'string'}
Execution harness: JavaScript
Recipe freeze marker
No recipe freeze time is recorded.