Agentic System for Global AI Product Competitive Intelligence
The rapid evolution of generative AI necessitates sophisticated competitive intelligence. This challenge tasks developers with building a multi-agent system using the OpenAI Agents SDK to autonomously monitor and analyze the global AI product landscape, inspired by headlines like ByteDance's Doubao 2.0 'agent era' upgrade. The system will leverage a team of specialized agents to gather data on new features, market positioning, and user sentiment for competing AI applications, synthesizing this information into actionable strategic insights.
What you are building
The core problem, expected build, and operating context for this challenge.
Build an OpenAI Agents SDK-based multi-agent system utilizing Claude 4 Sonnet, Portia AI, and Ludwig for autonomous competitive intelligence gathering and analysis.
How work is evaluated
Evaluation will assess the agent system's ability to accurately gather competitive intelligence, generate insightful analyses, and present findings effectively. It will measure the system's autonomy, tool utilization, and the quality of its strategic recommendations.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
AccurateDataExtraction
Verifies that key factual data (features, dates, names) is correctly extracted and reported.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
InsightfulAnalysis
Evaluates the depth and relevance of strategic implications and competitor comparisons.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
ToolUtilization
Checks if Portia AI and Ludwig were effectively integrated and used in the workflow.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
AnalysisCompleteness
Percentage of required analysis fields correctly populated. • target: 90 • range: 0-100
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
SentimentAccuracy
Accuracy of identified market sentiment compared to ground truth. • target: 0.85 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master the OpenAI Agents SDK for building role-based agent teams capable of dynamic tool use and multi-turn conversations.
Implement advanced function calling with Claude 4 Sonnet to enable agents to interact with external APIs and retrieve real-time data.
Design and integrate Portia AI as a specialized agent component for enhanced pattern recognition and anomaly detection within collected market data.
Leverage Ludwig for orchestrating the overall multi-step workflow, ensuring seamless transition between data collection, analysis, and reporting phases.
Utilize Prophet for real-time observability and tracing of agent actions, decisions, and output quality, facilitating debugging and optimization.
Develop a natural language interface using Hume AI to allow executive users to query the competitive intelligence system and receive concise, voice-synthesized summaries of findings.
Build a persistent memory mechanism for agents to retain context and learning across long-running analysis cycles, enhancing their analytical depth over time.
Reference links and supporting material
Sample data for 2 tasks
Challenge setup
Evaluation will assess the agent system's ability to accurately gather competitive intelligence, generate insightful analyses, and present findings effectively. It will measure the system's autonomy, tool utilization,...
Challenge input
{'product_name': 'NewAIProductX', 'launch_date_window': ['YYYY-MM-DD', 'YYYY-MM-DD']}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'analysis': {'product_name': str, 'key_features': list, 'market_sentiment': str, 'competitor_comparison': str, 'strategic_implications': str}, 'da...
- Verifies that key factual data (features, dates, names) is correctly extracted and reported.
- Evaluates the depth and relevance of strategic implications and competitor comparisons.
- Checks if Portia AI and Ludwig were effectively integrated and used in the workflow.
- AnalysisCompleteness target: 90
- 2 public reference cases
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- AccurateDataExtraction · Weight 1
- InsightfulAnalysis · Weight 1
- ToolUtilization · Weight 1
- AnalysisCompleteness · Weight 1
- SentimentAccuracy · Weight 1
Metrics
- AnalysisCompleteness · Target 90
- SentimentAccuracy · Target 0.85
Reference cases
- 2 public reference cases
Workflow tasks
- Task 1
Input: {'product_name': 'NewAIProductX', 'launch_date_window': ['YYYY-MM-DD', 'YYYY-MM-DD']}
Output: {'analysis': {'product_name': str, 'key_features': list, 'market_sentiment': str, 'competitor_comparison': str, 'strategic_implications': str}, 'data_sources': list}
Execution harness: Python
- Task 2
Input: {'product_name': 'EstablishedAIAppY', 'time_period': {'start_date': 'YYYY-MM-DD', 'end_date': 'YYYY-MM-DD'}, 'focus_event': 'major update'}
Output: {'sentiment_report': {'product_name': str, 'trend_overview': str, 'drivers': list, 'source_examples': list}, 'recommendations': list}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.