Prediction Market Intelligence System
The world of prediction markets like Kalshi and Polymarket offers unique insights into public sentiment and future events. This challenge tasks you with creating a multi-agent system using CrewAI that autonomously researches, analyzes, and synthesizes information about prediction market trends, trader behavior, and potential profitability patterns. The system should generate actionable market intelligence reports on specific market categories or recent events. You will define distinct roles for your agents (e.g., 'Market Data Analyst', 'Trend Spotter', 'Report Generator'), assign them specific goals, and enable them to collaborate effectively. The system will leverage Gemini 2.5 Pro for its advanced analytical capabilities and integrate tools for data retrieval and report generation, presenting findings through an intuitive interface.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop a CrewAI multi-agent system to generate insightful reports on prediction market trends and trader profitability using Gemini 2.5 Pro.
How work is evaluated
The CrewAI system will be evaluated on the quality, accuracy, and relevance of its generated market intelligence reports, as well as the efficiency of agent collaboration.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Report Format Adherence
Generated report adheres to the specified JSON structure.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Agent Workflow Completion
All defined agent tasks in the crew are completed.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Report Factual Accuracy
Percentage of statements in the report that are factually correct based on mock data. • target: 90 • range: 0-100
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Insight Generation Quality
Rating of the depth and novelty of insights provided. • target: 4 • range: 1-5
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
Collaboration Coherence
Smoothness and logical flow of agent handoffs and information exchange. • target: 4.5 • range: 1-5
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master CrewAI's core concepts: Agents, Tasks, Tools, and the Crew for building robust, collaborative multi-agent systems.
Implement specialized agents with distinct roles (e.g., MarketDataFetcher, BehavioralAnalyst, ReportSynthesizer) and assign them specific goals using CrewAI.
Design custom tools for agents to interact with a simulated prediction market API, fetching historical data, current odds, and trader statistics.
Leverage Gemini 2.5 Pro as the primary reasoning engine for complex market trend analysis and nuanced report generation within agent tasks.
Deploy and manage specialized analytical models or faster inference needs using RunPod for efficient real-time data processing.
Integrate Hume AI to provide an intuitive voice interface for users to submit queries and receive audio summaries of generated market intelligence reports.
Orchestrate sequential and parallel agent tasks to build a comprehensive market intelligence workflow.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
The CrewAI system will be evaluated on the quality, accuracy, and relevance of its generated market intelligence reports, as well as the efficiency of agent collaboration.
Challenge input
{'market_category': 'string', 'time_period': 'string'}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'title': 'string', 'summary': 'string', 'key_trends': ['string'], 'trader_behavior_insights': 'string', 'profitability_analysis': 'string', 'recom...
- Generated report adheres to the specified JSON structure.
- All defined agent tasks in the crew are completed.
- Percentage of statements in the report that are factually correct based on mock data. • target: 90 • range:...
- Report Factual Accuracy target: 90
- 2 public reference cases
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Report Format Adherence · Weight 1
- Agent Workflow Completion · Weight 1
- Report Factual Accuracy · Weight 1
- Insight Generation Quality · Weight 1
- Collaboration Coherence · Weight 1
Metrics
- Report Factual Accuracy · Target 90
- Insight Generation Quality · Target 4
- Collaboration Coherence · Target 4.5
Reference cases
- 2 public reference cases
Workflow tasks
- Task 1
Input: {'market_category': 'string', 'time_period': 'string'}
Output: {'title': 'string', 'summary': 'string', 'key_trends': ['string'], 'trader_behavior_insights': 'string', 'profitability_analysis': 'string', 'recommendations': ['string']}
Execution harness: Python
- Task 2
Input: {'task_description': 'string'}
Output: {'agent_interactions': [{'agent_name': 'string', 'task': 'string', 'result': 'string'}], 'final_output': 'string'}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.