TrendIdentification
Evaluate the LlamaIndex agent system's accuracy in identifying key AI trends, precision in sentiment analysis (including emotional cues from Hume AI), and the coherence of generated strategic insights.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
TrendIdentification
Assess the agent system's ability to identify emerging AI trends and associated developer sentiment from a corpus of tech news, forums, and simulated voice feedback.
JSON object with 'articles' (array of {title, content, source}), 'forum_posts' (array of {author, text, timestamp}), and 'voice_feedback_transcripts' (array of {speaker, transcript, audio_url_mock}).
JSON object with 'identified_trends' (array of {name, keywords, sentiment_score, confidence, supporting_evidence}), and 'overall_developer_sentiment' (string, e.g., 'positive', 'neutral', 'negative').