Build a Fraud Model Change Review and Shadow Operations Console

A fraud model governance and operations team.

Compare a candidate risk model with an existing model on chronological data. Give reviewers calibration, workload, subgroup uncertainty, and rollback evidence before simulated promotion.

Key: application_id + model_version + scoring_time.

Fields: probability, outcome, outcome_available_at, cohort_id, subgroup_metadata, threshold_version.

Protected attributes require governance controls. Aggregate metrics must not expose individual identities.

Freeze chronological training, calibration, and test partitions.

Compare shadow scores under fixed review-capacity policies.

Inspect calibration, subgroup uncertainty, and distribution shifts.

Export a model-change decision packet with rollback conditions.

A shadow comparison dashboard with calibration curves, queue effects, subgroup intervals, and approval controls.

A versioned shadow-scoring workflow.

A model-change record linking data, metrics, threshold decisions, and reviewer signoff.

Runnable local application, setup commands, and a dependency lockfile.

Versioned input data, data dictionary, acquisition instructions, and rights notes.

Calculation traces, source references, assumption records, and exported review artifacts.

A deterministic baseline and tests for normal, incomplete, contradictory, and failure-recovery cases.

A recorded complete workflow with a reproducible manifest and explicit limitations.

Thresholds are not tuned on the held-out test outcomes.

Compare precision-recall performance and workload at declared operating points.

Small subgroup samples produce uncertainty warnings, not fairness certification.

Reject fabricated claims that a public synthetic dataset proves bank production readiness.

The same probabilities can be poorly calibrated under shifted outcomes even when their rank order remains unchanged.

The attached starter files are prototype inputs. Expected starter answers are visible. They are not independent holdouts.

Full target: Acquire an authorized BAF variant or use synthetic tabular records. Retain the dataset datasheet and separate dataset variants from temporal holdouts.

Complete target contracts and expert review remain pending. Public observations are not historical release vintages.

scikit-learn, MLflow, Streamlit

Use an available model appropriate to the task. Record provider, model identifier, version or retrieval date, settings, cost, and latency. Use deterministic code for calculations. Models may extract, retrieve, or explain evidence. They must not invent missing records, override approval rules, or claim professional authority.

The build includes an operational review workflow, not only a leaderboard or a claim of fair production deployment.

No fabricated records, source citations, professional endorsements, or expert-adjudication claims.

No future-data leakage into dated decisions, model fitting, or historical valuation inputs.

No unauthorized payments, account restrictions, regulatory filings, or production changes.

No automated appraisal certification, tax-compliance determination, or unsupported fraud accusation.

No concealed uncertainty or silently invented inputs required for a numerical conclusion.

Correct domain calculations and justified methods with documented assumptions. Weight: 30.

Source lineage, temporal integrity, rights checks, and explicit synthetic-data labels. Weight: 25.

A working application with review, export, error handling, and reproducible state. Weight: 20.

Adversarial cases, uncertainty, missing-data behavior, and sensitivity analysis. Weight: 15.

Evidence-linked conclusions that distinguish facts, assumptions, and unresolved judgments. Weight: 10. Score each dimension from zero to two. Weighted score = sum(weight * score / 2). A failed safety gate blocks acceptance.