Public evaluation

CreditRiskAssessmentTask

Evaluation will focus on the accuracy and depth of financial analysis, the effectiveness of DSPy in optimizing LLM outputs, and the correct integration of financial data via MCP, alongside the intelligent use of adaptive thinking budgets.

Evaluation type

task based

Challenge

AI-Driven Financial Risk & Credit Scoring

Difficulty

Advanced

Rigor

Unspecified

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks

Benchmarks

Criteria

Task templates

Inputs and expected outputs.

Task 1

CreditRiskAssessmentTask

Perform a credit risk assessment for a mid-sized company based on provided financial statements and a simulated credit score.

Input format

{'company_name': 'string', 'annual_revenue': 'float', 'total_assets': 'float', 'total_liabilities': 'float', 'net_income': 'float', 'credit_score_sim': 'int'}

Output format

{'risk_category': 'low|medium|high|very_high', 'recommended_credit_limit': 'float', 'reasoning': 'string', 'key_financial_ratios': {'debt_to_equity': 'float', 'current_ratio': 'float'}}

Task 2

AdaptiveBudgetUtilizationTask

Demonstrate the use of adaptive thinking budgets for a complex 'what-if' financial scenario analysis, showing dynamic adjustment of LLM processing time.

Input format

{'scenario_description': 'string', 'base_revenue_projection': 'float', 'economic_downturn_impact': 'float'}

Output format

{'scenario_analysis_summary': 'string', 'budget_used_ms': 'int', 'decision_justification': 'string'}