Back to prompt library
Orchestrating Research Workflow with Prefect and CrewAI
Linked challenge: Collaborative Technical Research Crew
Prompt source
implementation54 lines9 sectionsNo variables1 code block
Define a Prefect flow that orchestrates the entire CrewAI research process for a given technical topic. The flow should include:
1. A `Researcher` task to perform initial web searches and store key findings in ChromaDB.
2. An `Analyst` task to retrieve findings from ChromaDB, synthesize them, and store structured insights back into ChromaDB.
3. A `Technical Writer` task to retrieve all relevant structured insights from ChromaDB and generate a comprehensive markdown report. This task should correspond to the `ResearchReportGeneration` evaluation task.
```python
# ... (previous agent and tool definitions)
from crewai import Task
from prefect import flow, task
# Define Tasks
task_research_topic = Task(
description=(
"Conduct comprehensive web research on the topic: '{topic}'. "
"Identify key historical milestones, major contributors, and technological advancements. "
"Store all relevant findings in ChromaDB using the 'store_research' tool."
),
expected_output='A summary of key research findings stored in ChromaDB.',
agent=researcher
)
task_analyze_findings = Task(
description=(
"Retrieve all research findings related to '{topic}' from ChromaDB. "
"Synthesize the information, identify patterns, and structure it into an outline. "
"Store the structured insights back into ChromaDB using the 'store_research' tool."
),
expected_output='A structured outline of insights related to the topic, stored in ChromaDB.',
agent=analyst
)
task_write_report = Task(
description=(
"Retrieve all structured insights and research data related to '{topic}' from ChromaDB. "
"Based on this, write a comprehensive, well-structured technical report in markdown format. "
"The report should include sections like Introduction, Key Concepts, Historical Context, Current State, Future Outlook, and Conclusion."
),
expected_output='A final markdown-formatted technical research report.',
agent=writer
)
@flow(name="Technical Research Flow")
def technical_research_flow(topic: str):
# Instantiate the CrewAI crew with the topic as context
tech_research_crew = Crew(
agents=[researcher, analyst, writer],
tasks=[task_research_topic, task_analyze_findings, task_write_report],
verbose=True,
process="sequential" # Or 'hierarchical' if using hierarchical agents
)
print(f"Starting research on: {topic}")
crew_result = tech_research_crew.kickoff(inputs={'topic': topic})
print(f"CrewAI research completed. Final report:\n{crew_result}")
return crew_result
# To run this flow in Prefect:
# 1. Ensure Prefect server is running
# 2. python your_script.py
# 3. technical_research_flow("The future of AI ethics and external audits.")
```