Build an Investigator Disagreement and Case Adjudication Portal

A fraud dataset curator coordinating qualified reviewers.

Collect independent investigator judgments, preserve disagreement, and manage adjudication evidence. Publish label provenance instead of pretending that ambiguous fraud cases have obvious ground truth.

Key: case_id + reviewer_id + review_round.

Fields: proposed_label, evidence_ids, confidence, abstention_reason, adjudicator_decision, label_version.

Store risk disposition, observed outcome, and intent as separate concepts.

Assign blinded independent case reviews.

Record labels, evidence, confidence, and abstentions.

Route disagreements to a documented adjudication workflow.

Export versioned labels with provenance and unresolved cases.

A blinded review queue, disagreement matrix, adjudication evidence panel, and dataset export controls.

A role-separated label review application.

A data card reporting reviewer participation, disagreements, and unresolved labels.

Runnable local application, setup commands, and a dependency lockfile.

Versioned input data, data dictionary, acquisition instructions, and rights notes.

Calculation traces, source references, assumption records, and exported review artifacts.

A deterministic baseline and tests for normal, incomplete, contradictory, and failure-recovery cases.

A recorded complete workflow with a reproducible manifest and explicit limitations.

Do not present fictional reviewer entries as expert participation.

Majority agreement is not treated as factual certainty.

Adversarial case families remain together within dataset partitions.

Disputed labels remain excluded or explicitly modeled as uncertain.

One reviewer changes a judgment because new evidence arrives. The system must retain both versions and reopen disagreement.

The attached starter files are prototype inputs. Expected starter answers are visible. They are not independent holdouts.

Full target: Author synthetic case pairs that differ in one evidence field. Use fictional reviewer decisions until qualified reviewers actually adjudicate the cases.

Complete target contracts and expert review remain pending. Public observations are not historical release vintages.

Pydantic, DuckDB, Streamlit

Use an available model appropriate to the task. Record provider, model identifier, version or retrieval date, settings, cost, and latency. Use deterministic code for calculations. Models may extract, retrieve, or explain evidence. They must not invent missing records, override approval rules, or claim professional authority.

No claims of bank endorsement, professional authorship, or expert-adjudicated labels without documented participation.

No fabricated records, source citations, professional endorsements, or expert-adjudication claims.

No future-data leakage into dated decisions, model fitting, or historical valuation inputs.

No unauthorized payments, account restrictions, regulatory filings, or production changes.

No automated appraisal certification, tax-compliance determination, or unsupported fraud accusation.

No concealed uncertainty or silently invented inputs required for a numerical conclusion.

Correct domain calculations and justified methods with documented assumptions. Weight: 30.

Source lineage, temporal integrity, rights checks, and explicit synthetic-data labels. Weight: 25.

A working application with review, export, error handling, and reproducible state. Weight: 20.

Adversarial cases, uncertainty, missing-data behavior, and sensitivity analysis. Weight: 15.

Evidence-linked conclusions that distinguish facts, assumptions, and unresolved judgments. Weight: 10. Score each dimension from zero to two. Weighted score = sum(weight * score / 2). A failed safety gate blocks acceptance.