Ambient AI Care Assistant for Nursing Facilities
Design and prototype an ambient AI care assistant utilizing Gemini 2.5 Pro, specifically tailored for skilled nursing facilities, focusing on real-time patient interaction summarization, proactive alert generation, and privacy-preserving integration with Electronic Health Records (EHR). This challenge will leverage Semantic Kernel's plugin architecture for seamless tool integration and emphasize hybrid reasoning to differentiate between instant summarization needs and deep analysis for critical alerts, ensuring data security and ethical considerations are paramount.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an ambient AI care assistant prototype using Gemini 2.5 Pro and Semantic Kernel for real-time patient interaction summarization and alert generation in nursing facilities, with strong privacy integration.
How work is evaluated
The evaluation will assess the ambient AI care assistant's ability to accurately summarize patient interactions, generate relevant and timely alerts, and demonstrate secure integration with a mock EHR system via Semantic Kernel plugins, with a focus on privacy-preserving techniques.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Develop a hybrid reasoning system to generate instant summaries and deep, contextualized alerts.
Implement real-time audio processing and summarization using Gemini 2.5 Pro for patient interactions.
Create a user-friendly interface for care staff to review summaries and alerts.
Design and integrate Semantic Kernel plugins for secure, privacy-preserving access to simulated EHR systems.
Address ethical AI considerations and data privacy (HIPAA compliance) within the system design.
Reference links and supporting material
Sample data for 1 tasks
Challenge setup
The evaluation will assess the ambient AI care assistant's ability to accurately summarize patient interactions, generate relevant and timely alerts, and demonstrate secure integration with a mock EHR system via Seman...
Challenge input
{'transcript': 'text', 'patient_id': 'str', 'mock_ehr_data': {'patient_id': {'name': 'str', 'medications': ['str'], 'allergies': ['str'], 'care_pla...
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'summary': 'text', 'alerts': [{'type': 'str', 'description': 'str', 'severity': 'str', 'ehr_context_queried': 'bool'}], 'ehr_updates': [{'action':...
- The system generates at least one high-severity alert for a critical event in the transcript.
- The system successfully queries the mock EHR and indicates context was used (ehr_context_queried: true for...
- SummaryAccuracy target: 0.8
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- The system generates at least one high-severity alert for a critical event in the transcript.
- The system successfully queries the mock EHR and indicates context was used (ehr_context_queried: true for relevant alerts).
Metrics
- SummaryAccuracy · Target 0.8
- AlertPrecision · Target 0.85
Workflow tasks
- Task 1
Input: {'transcript': 'text', 'patient_id': 'str', 'mock_ehr_data': {'patient_id': {'name': 'str', 'medications': ['str'], 'allergies': ['str'], 'care_plan': 'text'}}}
Output: {'summary': 'text', 'alerts': [{'type': 'str', 'description': 'str', 'severity': 'str', 'ehr_context_queried': 'bool'}], 'ehr_updates': [{'action': 'str', 'data': 'json'}]}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.