Skip to main content
Loading navigation
Back to prompt library

Build a Regional Policy Counterfactual Comparison Studio — Implementation planning and evidence-linked development

Linked challenge: Build a Regional Policy Counterfactual Comparison Studio

Prompt source

Implementation planning and evidence-linked development51 lines16 sectionsNo variables28 checklist items
## User
A regional policy research team studying one treated location.

## Build
Construct and inspect a synthetic comparison region for a policy study. Track donor eligibility, pre-intervention fit, placebo results, and spillover sensitivity.

## Input Contract
- Key: region_id + period + data_snapshot.
- Fields: outcome, predictors, intervention_date, donor_eligibility, spillover_flag.
- Freeze donor selection and fitting inputs before post-intervention outcomes are inspected.

## Workflow
1. Document the intervention and eligible donor pool.
2. Fit constrained donor weights on pre-intervention data.
3. Compare post-intervention gaps with placebo and exclusion exercises.
4. Export the fitted counterfactual and limitations on inference.

## Interface
A donor selection grid, weight chart, observed-versus-counterfactual plot, and placebo comparison panel.

## Deliverables
- A reproducible constrained-weight model.
- A report retaining excluded donors and pre-fit diagnostics.
- Runnable local application, setup commands, and a dependency lockfile.
- Versioned input data, data dictionary, acquisition instructions, and rights notes.
- Calculation traces, source references, assumption records, and exported review artifacts.
- A deterministic baseline and tests for normal, incomplete, contradictory, and failure-recovery cases.
- A recorded complete workflow with a reproducible manifest and explicit limitations.

## Acceptance Criteria
- Weights obey the declared nonnegativity and sum constraints.
- Post-treatment outcomes do not enter the fitting objective.
- Poor pre-fit or likely spillovers prevent a strong causal claim.
- Placebo comparisons disclose their eligibility and filtering rules.

## Contrast Case
Changing donor weights changes the counterfactual even when the treated outcome remains fixed.

## Dataset Scope
The attached starter files are prototype inputs. Expected starter answers are visible. They are not independent holdouts.

Full target: Use a synthetic regional panel with one intervention and several potential donors. Public country indicators can supply context, not automatic causal comparability.

Complete target contracts and expert review remain pending. Public observations are not historical release vintages.

## Tools
SciPy, pandas, Plotly

Use an available model appropriate to the task. Record provider, model identifier, version or retrieval date, settings, cost, and latency. Use deterministic code for calculations. Models may extract, retrieve, or explain evidence. They must not invent missing records, override approval rules, or claim professional authority.

## Constraints
A visually close fit alone does not establish a valid causal counterfactual.

- No fabricated records, source citations, professional endorsements, or expert-adjudication claims.
- No future-data leakage into dated decisions, model fitting, or historical valuation inputs.
- No unauthorized payments, account restrictions, regulatory filings, or production changes.
- No automated appraisal certification, tax-compliance determination, or unsupported fraud accusation.
- No concealed uncertainty or silently invented inputs required for a numerical conclusion.

## Scoring
- Correct domain calculations and justified methods with documented assumptions. Weight: 30.
- Source lineage, temporal integrity, rights checks, and explicit synthetic-data labels. Weight: 25.
- A working application with review, export, error handling, and reproducible state. Weight: 20.
- Adversarial cases, uncertainty, missing-data behavior, and sensitivity analysis. Weight: 15.
- Evidence-linked conclusions that distinguish facts, assumptions, and unresolved judgments. Weight: 10.
Score each dimension from zero to two. Weighted score = sum(weight * score / 2). A failed safety gate blocks acceptance.