Real-time Social Listening Assistant
This challenge tasks developers with building an advanced AI assistant that monitors a simulated stream of friends' activities (e.g., music listening, article reading). The assistant must leverage a powerful custom LLM to generate personalized, conversational summaries and recommendations, and use a dedicated memory system to maintain context and user preferences over time. This system goes beyond simple notifications, aiming to create engaging, context-aware interactions. It requires real-time data processing, sophisticated natural language generation, and intelligent memory management to deliver a truly personalized experience.
What you are building
The core problem, expected build, and operating context for this challenge.
Build a real-time conversational AI assistant that offers personalized insights and recommendations using a custom LLM and long-term memory.
How work is evaluated
The evaluation module assesses the assistant's ability to process simulated real-time events, provide contextually relevant and personalized responses, and maintain a coherent conversational flow. It will check for accuracy in summaries and the relevance of recommendations based on memory and current activity.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Master conversational AI design using Voiceflow to create intuitive multi-turn interactions and manage state within the assistant's flow.
Implement real-time data ingestion and processing pipelines for simulated friend activity updates, focusing on low-latency data handling.
Integrate MemGPT for persistent context and user preference storage, enabling deep personalization across sessions and interactions.
Utilize Hunyuan (or a similar custom-trained LLM via API) for generating natural language summaries of activities and creative, personalized recommendations.
Deploy the Hunyuan model and the associated assistant logic using BentoML Cloud for scalable, low-latency inference and API serving.
Develop strategies for dynamic prompt engineering that adapts to changing user context, real-time events, and memory-derived insights.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The evaluation module assesses the assistant's ability to process simulated real-time events, provide contextually relevant and personalized responses, and maintain a coherent conversational flow. It will check for ac...
Challenge input
{'event_type': 'music_listening', 'user_id': 'friend123', 'song': 'Track Name', 'artist': 'Artist Name', 'timestamp': 'ISO8601'}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'summary': 'string', 'relevance_score': 'float'}
- Generated summary accurately reflects the input activity.
- Recommendation includes a coherent justification linking to memory or current context.
- Average Personalization Score target: 0.85
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Generated summary accurately reflects the input activity.
- Recommendation includes a coherent justification linking to memory or current context.
Metrics
- Average Personalization Score · Target 0.85
- Conversational Coherence Score (Semantic Similarity) · Target 0.8
Workflow tasks
- Task 1
Input: {'event_type': 'music_listening', 'user_id': 'friend123', 'song': 'Track Name', 'artist': 'Artist Name', 'timestamp': 'ISO8601'}
Output: {'summary': 'string', 'relevance_score': 'float'}
Execution harness: Python
- Task 2
Input: {'user_id': 'current_user', 'latest_activity_summary': 'string', 'conversation_history': 'list of dicts'}
Output: {'recommendation': 'string', 'justification': 'string', 'personalization_score': 'float'}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.