Skip to main content
Challenge

Build a Yente Entity Screening Task Auditor

Audit investigative shift logs to ensure all mentioned counterparties were properly formatted as FollowTheMoney entities and screened via the yente match API. All outputs are mock reviews.

CybersecurityHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Audit investigative shift logs to ensure all mentioned counterparties were properly formatted as FollowTheMoney entities and screened via the yente match API. All outputs are mock reviews.

Shift continuity supervisor

Ingest unstructured shift notes detailing investigative leads.

Extract counterparty names and map them to FollowTheMoney entity schemas.

Verify if the shift ledger includes a recorded yente API match operation for each entity.

Generate a discrepancy report highlighting entities mentioned but never screened.

Dashboard displaying unscreened entity alerts and screening continuity metrics.

Construct local synthetic JSON files containing shift notes with entity mentions, alongside yente API request logs. Use simplified synthetic app records, not complete official schema files. Starter examples below are synthetic. They are not downloaded records.

OpenSanctions data toolkit Applies FollowTheMoney schema standards to structure extracted entities for yente match verification.

mastra-ai, pydantic, sqlite3. Provides stateful agent orchestration to ingest shift notes, extract entities, and identify missing yente screening events. Local or hosted LLM with strong logical deduction to map stated entities against yente action logs.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Extracts all counterparty entities from unstructured shift notes.

Maps extracted names to the FollowTheMoney schema.

Generates an accurate discrepancy report listing entities that lack a corresponding yente screening action.

```json [ { "input": { "shift_id": "SHIFT-01", "analyst_logs": [ { "time": "09:00", "content": "Will review alerts for Acme Corp." }, { "time": "11:00", "content": "Screened Global Trade LLC via yente, cleared." } ] }, "expected_output": { "shift_id": "SHIFT-01", "extracted_entities": [ "Acme Corp", "Global Trade LLC" ], "screened_entities": [ "Global Trade LLC" ], "dropped_screenings": [ "Acme Corp" ] } }, { "input": { "shift_id": "SHIFT-02", "analyst_logs": [ { "time": "14:00", "content": "Need to check Oceanic Ltd against sanctions." }, { "time": "15:30", "content": "yente match API returned no hits for Oceanic Ltd." } ] }, "expected_output": { "shift_id": "SHIFT-02", "extracted_entities": [ "Oceanic Ltd" ], "screened_entities": [ "Oceanic Ltd" ], "dropped_screenings": [] } } ] ```

Does not autonomously alter alert systems; only flags potential screening gaps for human supervisor review. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Does not autonomously alter alert systems; only flags potential screening gaps for human supervisor review. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Extracts all counterparty entities from unstructured shift notes. Maps extracted names to the FollowTheMoney schema. Generates an accurate discrepancy report listing entities that lack a corresponding yente screening action.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest unstructured shift notes detailing investigative leads. Extract counterparty names and map them to FollowTheMoney entity schemas. Verify if the shift ledger includes a recorded yente API match operation for each entity. Generate a discrepancy report highlighting entities mentioned but never screened.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Extracts all counterparty entities from unstructured shift notes. Maps extracted names to the FollowTheMoney schema. Generates an accurate discrepancy report listing entities that lack a corresponding yente screening action.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

Dashboard displaying unscreened entity alerts and screening continuity metrics.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Does not autonomously alter alert systems; only flags potential screening gaps for human supervisor review.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest unstructured shift notes detailing investigative leads.

  • Extract counterparty names and map them to FollowTheMoney entity schemas.

  • Verify if the shift ledger includes a recorded yente API match operation for each entity.

  • Generate a discrepancy report highlighting entities mentioned but never screened.

Resources and assets

Reference links and supporting material

Dataset notes

Construct local synthetic JSON files containing shift notes with entity mentions, alongside yente API request logs. Use simplified synthetic app records, not complete official schema files. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

Mastra AI

TypeScript agent framework.

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest unstructured shift notes detailing investigative leads. Extract counterparty names and map them to F...
  • Extracts all counterparty entities from unstructured shift notes. Maps extracted names to the FollowTheMone...
  • Dashboard displaying unscreened entity alerts and screening continuity metrics.
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 2 configured tools

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Optional
Orchestration
  • Mastra AI · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a Yente Entity Screening Task Auditor