Build a Policy Evidence Replication and Transportability Desk

A policy evidence team preparing a recommendation review.

Compare studies that appear to answer the same policy question. Reconcile populations, outcomes, horizons, and methods before considering whether results transfer to a new setting.

Key: study_id + estimand_id + result_version.

Fields: population, intervention, comparator, outcome_unit, horizon, estimate, uncertainty, design_limitations.

Do not merge unlike outcomes or horizons into a single average effect.

Extract and verify study definitions against supplied records.

Replicate available arithmetic and flag unavailable analyses.

Compare target-population overlap and transport assumptions.

Export an evidence matrix and bounded recommendation draft.

A study comparison matrix with estimand filters, replication status, and applicability review notes.

An evidence registry with structured estimand definitions.

A synthesis report separating reproduced calculations from unsupported generalization.

Runnable local application, setup commands, and a dependency lockfile.

Versioned input data, data dictionary, acquisition instructions, and rights notes.

Calculation traces, source references, assumption records, and exported review artifacts.

A deterministic baseline and tests for normal, incomplete, contradictory, and failure-recovery cases.

A recorded complete workflow with a reproducible manifest and explicit limitations.

Incompatible outcome units block pooling.

Study quality is not inferred from sample size alone.

Target-population weighting is labeled conditional on transport assumptions.

Unavailable data produce a replication limitation, not an invented result.

Subgroup effects remain unchanged while the target population mix changes. A conditional transported average changes accordingly.

The attached starter files are prototype inputs. Expected starter answers are visible. They are not independent holdouts.

Full target: Author six fictional study records with reproducible summary tables and contrasting estimands. Use public population metadata only as contextual evidence.

Complete target contracts and expert review remain pending. Public observations are not historical release vintages.

pandas, Pydantic, Jinja2

Use an available model appropriate to the task. Record provider, model identifier, version or retrieval date, settings, cost, and latency. Use deterministic code for calculations. Models may extract, retrieve, or explain evidence. They must not invent missing records, override approval rules, or claim professional authority.

Arithmetic reweighting does not establish causal transportability.

No fabricated records, source citations, professional endorsements, or expert-adjudication claims.

No future-data leakage into dated decisions, model fitting, or historical valuation inputs.

No unauthorized payments, account restrictions, regulatory filings, or production changes.

No automated appraisal certification, tax-compliance determination, or unsupported fraud accusation.

No concealed uncertainty or silently invented inputs required for a numerical conclusion.

Correct domain calculations and justified methods with documented assumptions. Weight: 30.

Source lineage, temporal integrity, rights checks, and explicit synthetic-data labels. Weight: 25.

A working application with review, export, error handling, and reproducible state. Weight: 20.

Adversarial cases, uncertainty, missing-data behavior, and sensitivity analysis. Weight: 15.

Evidence-linked conclusions that distinguish facts, assumptions, and unresolved judgments. Weight: 10. Score each dimension from zero to two. Weighted score = sum(weight * score / 2). A failed safety gate blocks acceptance.