Orchestrate Data Ingestion with Prefect and Report Generation

implementationChallenge

Prompt Content

Design a Prefect flow to orchestrate the data ingestion and preprocessing for your CRE analysis. This flow should fetch raw data (simulated from a file or API), clean it, and make it available for your AutoGen agents. Additionally, outline how the AutoGen system's final investment report can be automatically formatted and published (e.g., to a PDF or a web dashboard like Plotly Dash) once the agents complete their analysis. Show how Prefect would trigger the AutoGen analysis and collect its output.

```python
from prefect import flow, task
# import pandas as pd

# @task
# def fetch_raw_cre_data(source: str) -> pd.DataFrame:
#    # Simulate fetching data
#    return pd.DataFrame({'location': ['Austin', 'Seattle'], 'type': ['retail', 'office'], 'yield': [0.05, 0.04]})

# @task
# def preprocess_data(df: pd.DataFrame) -> dict:
#    # Simulate preprocessing
#    return df.set_index('location').to_dict('index')

# @flow
# def cre_analysis_pipeline(query: str):
#    raw_data = fetch_raw_cre_data("external_api")
#    processed_data = preprocess_data(raw_data)

#    # Trigger AutoGen here, passing processed_data as context or tool input
#    # final_report = autogen_manager.run_analysis(query, processed_data)

#    # publish_report(final_report)

# if __name__ == "__main__":
#    cre_analysis_pipeline(query="Analyze retail properties.")
```

Try this prompt

Open the workspace to execute this prompt with free credits, or use your own API keys for unlimited usage.

Usage Tips

Copy the prompt and paste it into your preferred AI tool (Claude, ChatGPT, Gemini)

Customize placeholder values with your specific requirements and context

For best results, provide clear examples and test different variations