Financial Proxy Analyst AI
This challenge focuses on developing an advanced AI system capable of ingesting and analyzing complex corporate proxy statements. The goal is to extract key financial and governance data, synthesize insights, and generate well-justified voting recommendations or executive summaries. This system must demonstrate sophisticated document understanding and output generation, paired with a robust evaluation framework. Developers will focus on precisely extracting structured information from unstructured legal text, using a powerful LLM to reason and synthesize, and employing an MLOps platform to ensure the quality and reliability of the AI's financial analysis outputs.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop an AI system to analyze complex financial documents, generate concise summaries and justified recommendations, with robust evaluation pipelines.
How work is evaluated
The evaluation module systematically assesses the AI system's performance in three key areas: accuracy of structured data extraction, quality and coherence of executive summaries, and the logical soundness and justification of generated recommendations. This will involve comparing AI outputs against expert-labeled ground truth.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Master parsing and pre-processing of complex PDF documents (e.g., corporate proxy statements) using LlamaIndex's document loaders and advanced parsing tools to prepare content for LLM ingestion.
Implement advanced prompt engineering techniques with Claude Opus 4.5, leveraging its long-context window for structured data extraction, multi-hop reasoning, and sophisticated synthesis from financial and governance text.
Design a system to generate clear, concise executive summaries and justified voting recommendations based on extracted financial data, governance proposals, and predefined policy criteria.
Integrate with enterprise data systems using Paragon to securely retrieve proxy statements from a document repository and publish generated analyses or recommendations to a corporate dashboard.
Build and utilize an MLflow evaluation pipeline to objectively assess the accuracy, completeness, and justification quality of the AI-generated outputs against a curated set of ground truth or expert-annotated examples.
Explore techniques for validating generated outputs against financial benchmarks, regulatory guidelines, and company-specific voting policies to ensure compliance and strategic alignment.
Reference links and supporting material
Sample data for 3 tasks
Challenge setup
The evaluation module systematically assesses the AI system's performance in three key areas: accuracy of structured data extraction, quality and coherence of executive summaries, and the logical soundness and justifi...
Challenge input
{'document_text': 'string', 'target_entities': ['entity1', 'entity2']}
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{'extracted_data': {'entity_name': 'extracted_value'}, 'accuracy_score': 'float'}
- Ensures essential financial and governance entities (e.g., CEO compensation, key dates) are always extracte...
- Checks if the recommendation's justification directly references information from the provided document and...
- Average F1 Score For Named Entity Recognition (NER) target: 0.88
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Ensures essential financial and governance entities (e.g., CEO compensation, key dates) are always extracted correctly.
- Checks if the recommendation's justification directly references information from the provided document and policy guidelines.
Metrics
- Average F1-score for Named Entity Recognition (NER) · Target 0.88
- Average ROUGE-L Score for Summaries · Target 0.7
- Recommendation Consistency Rate · Target 0.9
Workflow tasks
- Task 1
Input: {'document_text': 'string', 'target_entities': ['entity1', 'entity2']}
Output: {'extracted_data': {'entity_name': 'extracted_value'}, 'accuracy_score': 'float'}
Execution harness: Python
- Task 2
Input: {'document_text': 'string', 'summary_length_target': 'int'}
Output: {'summary_text': 'string', 'rouge_score': 'float', 'bertscore': 'float'}
Execution harness: Python
- Task 3
Input: {'document_text': 'string', 'proposal_id': 'string', 'policy_guidelines': 'string'}
Output: {'recommendation': 'string', 'justification': 'string', 'validity_score': 'float'}
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.