Build a Frictionless Tabular Transaction Data Validator

Validate bulk tabular transaction datasets using the frictionless-py framework. Detect missing fields, type constraints, and duplicated headers to ensure data quality before AML pipeline ingestion.

Data Engineer

Ingest raw tabular CSV transaction files and predefined Frictionless Table Schema metadata.

Execute the frictionless-py validation engine to verify column types and foreign key constraints.

Parse the generated unified validation report to extract specific row and cell-level errors.

Generate a filtered error log outlining required tabular data repairs for data engineers.

A data quality dashboard highlighting specific invalid rows, missing fields, and type constraint violations.

Create synthetic messy CSV transaction extracts alongside a strict JSON Frictionless Table Schema specification. Starter examples below are synthetic. They are not downloaded records.

Frictionless data validation Employ frictionless-py validation primitives and schema resource descriptors to inspect tabular data as per source excerpt 3.

pydantic-ai, frictionless, pandas. Pydantic AI enforces strict typing for incoming tabular records and guarantees structured error report payloads. Standard inference model for parsing custom validation terms via hosted API.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Executes frictionless-py schema validation against incoming CSV tables.

Accurately maps cell-level errors to their corresponding schema constraints.

Generates a unified JSON validation report summarizing repair requirements.

```json [ { "input": { "dataset_id": "TX-BATCH-01", "schema_strict": true, "rows": [ { "id": "1", "amount": "100.50" }, { "id": "2", "amount": "invalid_string" } ], "note": "These are simplified synthetic app records, not complete official schema files." }, "expected_output": { "dataset_id": "TX-BATCH-01", "is_valid": false, "errors": [ { "row": 2, "field": "amount", "error_type": "type-error" } ] } }, { "input": { "dataset_id": "TX-BATCH-02", "schema_strict": true, "rows": [ { "id": "3", "amount": "250.00" } ], "note": "These are simplified synthetic app records, not complete official schema files." }, "expected_output": { "dataset_id": "TX-BATCH-02", "is_valid": true, "errors": [] } } ] ```

Processes synthetic transaction records. Generates simulated validation reports without interacting with live production AML databases. Use mock external writes. Require human review before any real deployment.