Build a Forecast-to-Actual Valuation Assumption Review System
Compare archived operating forecasts with later outcomes. Attribute misses to assumptions, acquisitions, currency, and reporting changes before revising a valuation model.
What you are building
The core problem, expected build, and operating context for this challenge.
A corporate finance team reviewing forecast quality.
Compare archived operating forecasts with later outcomes. Attribute misses to assumptions, acquisitions, currency, and reporting changes before revising a valuation model.
Key: forecast_version + target_period + business_scope + metric.
Fields: forecast_asof, forecast_value, actual_value, actual_release, bridge_adjustment, adjustment_source.
Keep forecast vintages immutable. Compare like-for-like business scope.
Freeze forecast versions and their assumptions.
Align actuals to the forecast business scope.
Separate raw error from documented comparability adjustments.
Export a model-change proposal and an approval history.
A forecast vintage matrix with scope adjustments, error attribution, and a revision approval queue.
An immutable forecast review store.
An assumption-change report with raw and adjusted forecast errors.
Runnable local application, setup commands, and a dependency lockfile.
Versioned input data, data dictionary, acquisition instructions, and rights notes.
Calculation traces, source references, assumption records, and exported review artifacts.
A deterministic baseline and tests for normal, incomplete, contradictory, and failure-recovery cases.
A recorded complete workflow with a reproducible manifest and explicit limitations.
Do not overwrite an old forecast with the latest plan.
Acquisition bridges require evidence or explicit synthetic assumptions.
Adjusted and unadjusted errors remain visible together.
Later outcomes cannot alter the original forecast information set.
The actual total is unchanged, but an acquired business adds revenue outside forecast scope. Comparable forecast error must change.
The attached starter files are prototype inputs. Expected starter answers are visible. They are not independent holdouts.
Full target: Use dated fictional management forecasts and public actuals. Supply acquisition, currency, and accounting-change bridges explicitly rather than infer unavailable management detail.
Complete target contracts and expert review remain pending. Public observations are not historical release vintages.
DuckDB, pandas, MLflow
Use an available model appropriate to the task. Record provider, model identifier, version or retrieval date, settings, cost, and latency. Use deterministic code for calculations. Models may extract, retrieve, or explain evidence. They must not invent missing records, override approval rules, or claim professional authority.
A forecast miss does not establish misconduct or prove that the original valuation was unreasonable.
No fabricated records, source citations, professional endorsements, or expert-adjudication claims.
No future-data leakage into dated decisions, model fitting, or historical valuation inputs.
No unauthorized payments, account restrictions, regulatory filings, or production changes.
No automated appraisal certification, tax-compliance determination, or unsupported fraud accusation.
No concealed uncertainty or silently invented inputs required for a numerical conclusion.
Correct domain calculations and justified methods with documented assumptions. Weight: 30.
Source lineage, temporal integrity, rights checks, and explicit synthetic-data labels. Weight: 25.
A working application with review, export, error handling, and reproducible state. Weight: 20.
Adversarial cases, uncertainty, missing-data behavior, and sensitivity analysis. Weight: 15.
Evidence-linked conclusions that distinguish facts, assumptions, and unresolved judgments. Weight: 10. Score each dimension from zero to two. Weighted score = sum(weight * score / 2). A failed safety gate blocks acceptance.
Participation guide
A forecast miss does not establish misconduct or prove that the original valuation was unreasonable. No fabricated records, source citations, professional endorsements, or expert-adjudication claims. No future-data leakage into dated decisions, model fitting, or historical valuation inputs. No unauthorized payments, account restrictions, regulatory filings, or production changes. No automated appraisal certification, tax-compliance determination, or unsupported fraud accusation. No concealed uncertainty or silently invented inputs required for a numerical conclusion.
How work is evaluated
100-point weighted review with safety gates. Visible starter cases do not establish full application correctness.
An immutable forecast review store. An assumption-change report with raw and adjusted forecast errors. Runnable local application, setup commands, and a dependency lockfile. Versioned input data, data dictionary, acquisition instructions, and rights notes. Calculation traces, source references, assumption records, and exported review artifacts. A deterministic baseline and tests for normal, incomplete, contradictory, and failure-recovery cases. A recorded complete workflow with a reproducible manifest and explicit limitations.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
What you should walk away with
Freeze forecast versions and their assumptions.
Align actuals to the forecast business scope.
Separate raw error from documented comparability adjustments.
Export a model-change proposal and an approval history.
Reference links and supporting material
1 attached dataset file(s), including two visible synthetic starter cases. Complete target data remains pending.
Challenge setup
Configuration summary for this challenge.
Challenge input
Input details are not configured.
Agent execution
No agent tool is listed for this challenge.
Evaluated output
No output checks are configured.
- Rubric dimension 1
- Rubric dimension 2
- Rubric dimension 3
- 2 public reference cases
- No runtime settings are configured.
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Rubric dimension 1 · Weight 30
- Rubric dimension 2 · Weight 25
- Rubric dimension 3 · Weight 20
- Rubric dimension 4 · Weight 15
- Rubric dimension 5 · Weight 10
Reference cases
- 2 public reference cases
Workflow tasks
No workflow tasks are configured.
Recipe freeze marker
No recipe freeze time is recorded.