Complex Recommendation Generation
The evaluation will assess the recommendation system's ability to handle complex queries, retrieve obscure information, generate creative and relevant recommendations, and demonstrate effective agent collaboration and tool usage. Focus on relevance, creativity, factual accuracy, and the depth of reasoning.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
Complex Recommendation Generation
Evaluate the system's ability to generate a personalized, multi-faceted recommendation for a user based on a complex query involving obscure preferences and inter-domain connections.
{ "user_profile": { "age": 30, "mood": "reflective", "fav_genres": ["indie folk", "ambient electronic"], "recent_activities": ["reading philosophy", "stargazing"] }, "query": "I'm looking for a new album that combines introspective lyrics with atmospheric soundscapes, something that pairs well with late-night contemplation and has a subtle nod to early 2000s indie electronica, perhaps from an artist not widely known." }
{ "recommendation_title": "string", "artist": "string", "genre_tags": ["string"], "reasoning": "string (detailed explanation of why this recommendation fits the query, referencing obscure elements and user profile)", "external_sources_used": [{"tool_name": "string", "data_retrieved": "string"}] }