Skip to main content
Challenge

Build a Multi-Source Watchlist Lineage and Adjudication Dispute Arbiter

Ingest divergent sanctions screening dispositions from multiple junior reviewers across distinct watchlist feeds. Map entity provenance to FollowTheMoney schema models and compile evidence arbitration packages for chief compliance officers.

CybersecurityHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Ingest divergent sanctions screening dispositions from multiple junior reviewers across distinct watchlist feeds. Map entity provenance to FollowTheMoney schema models and compile evidence arbitration packages for chief compliance officers.

Sanctions Review Escalation Lead

Ingest conflicting analyst screening adjudications containing individual reviewer rationale and confidence scores.

Retrieve upstream entity records and lineage citations from local FollowTheMoney entities via a simulated yente API.

Detect discrepancies in underlying sanction program tags, aliases, and source update timestamps across conflicting reviews.

Synthesize an arbitration dossier compiling provenance lineage and consensus recommendation drafts for senior auditor sign-off.

A multi-column discrepancy dashboard showing reviewer adjudications alongside underlying FollowTheMoney entity source provenance trees.

Normalized JSON review decision logs cross-referenced with synthetic FollowTheMoney entity payloads hosted on a local test mock server. Starter examples below are synthetic. They are not downloaded records.

OpenSanctions data toolkit Leverages FollowTheMoney schema entity attributes, sanctions program properties, and yente matchmaking endpoint structures.

dspy, pydantic, fastapi. DSPy compiles robust prompt pipelines that objectively synthesize conflicting analyst rationale into standardized dispute evidence briefs. Local or hosted LLM evaluated with teleprompters for zero-shot text summarization and factual consistency.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Extracts source provenance identifiers and dataset publication timestamps for each contested watchlist hit.

Identifies exact root causes of reviewer divergence including alias coverage or stale watchlist versions.

Outputs an immutable arbitration dossier with deterministic discrepancy scoring and zero unassigned review items.

```json [ { "input": { "entity_id": "ftm-person-8841", "reviews": [ { "analyst": "analyst_a", "disposition": "FALSE_POSITIVE", "notes": "Birth year differs by 10 years" }, { "analyst": "analyst_b", "disposition": "POTENTIAL_MATCH", "notes": "Matches alternate spelling and nationality" } ] }, "expected_output": { "dispute_status": "ESCALATED_FOR_ARBITRATION", "provenance_dataset": "ofac_sdn", "root_cause": "BIRTH_DATE_ALIAS_CONFLICT", "provenance_timestamp": "2024-03-01T00:00:00Z", "consensus_score": 0.5, "recommended_review_step": "REQUEST_PASSPORT_COPY" } }, { "input": { "entity_id": "ftm-corp-1029", "reviews": [ { "analyst": "analyst_a", "disposition": "TRUE_POSITIVE", "notes": "Exact registration number match" } ] }, "expected_output": { "dispute_status": "UNANIMOUS_CONSISTENT", "provenance_dataset": "eu_fsf", "root_cause": "NONE", "provenance_timestamp": "2024-02-15T00:00:00Z", "consensus_score": 1, "recommended_review_step": "APPLY_REGULATORY_RESTRICTION" } } ] ```

Performs advisory compliance hit adjudication without autonomous account closures, asset freezes, or live law enforcement reports. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Performs advisory compliance hit adjudication without autonomous account closures, asset freezes, or live law enforcement reports. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Extracts source provenance identifiers and dataset publication timestamps for each contested watchlist hit. Identifies exact root causes of reviewer divergence including alias coverage or stale watchlist versions. Outputs an immutable arbitration dossier with deterministic discrepancy scoring and zero unassigned review items.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest conflicting analyst screening adjudications containing individual reviewer rationale and confidence scores. Retrieve upstream entity records and lineage citations from local FollowTheMoney entities via a simulated yente API. Detect discrepancies in underlying sanction program tags, aliases, and source update timestamps across conflicting reviews. Synthesize an arbitration dossier compiling provenance lineage and consensus recommendation drafts for senior auditor sign-off.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Extracts source provenance identifiers and dataset publication timestamps for each contested watchlist hit. Identifies exact root causes of reviewer divergence including alias coverage or stale watchlist versions. Outputs an immutable arbitration dossier with deterministic discrepancy scoring and zero unassigned review items.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

A multi-column discrepancy dashboard showing reviewer adjudications alongside underlying FollowTheMoney entity source provenance trees.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Performs advisory compliance hit adjudication without autonomous account closures, asset freezes, or live law enforcement reports.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest conflicting analyst screening adjudications containing individual reviewer rationale and confidence scores.

  • Retrieve upstream entity records and lineage citations from local FollowTheMoney entities via a simulated yente API.

  • Detect discrepancies in underlying sanction program tags, aliases, and source update timestamps across conflicting reviews.

  • Synthesize an arbitration dossier compiling provenance lineage and consensus recommendation drafts for senior auditor sign-off.

Resources and assets

Reference links and supporting material

Dataset notes

Normalized JSON review decision logs cross-referenced with synthetic FollowTheMoney entity payloads hosted on a local test mock server. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

DSPy

Prompt and LM program optimization framework.

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest conflicting analyst screening adjudications containing individual reviewer rationale and confidence...
  • Extracts source provenance identifiers and dataset publication timestamps for each contested watchlist hit....
  • A multi-column discrepancy dashboard showing reviewer adjudications alongside underlying FollowTheMoney ent...
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 3 configured tools

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • DSPy · Optional
Training Infra
  • DSPy · Optional
Orchestration
  • DSPy · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Multi-Source Watchlist Lineage and Adjudication Dispute Arbiter