Skip to main content
Back to challenges
Challenge

Gig Worker Grievance Redressal Workflow using Mastra AI

Service platforms require automated, audit-trailed resolution workflows for service provider disputes and ratings appeals. Build a stateful grievance handler using Mastra AI with integrated memory vectors and rule tools to resolve standard worker dispute tickets with 90%+ resolution agreement against human judge benchmarks.

Business OperationsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build an automated gig economy grievance triage and resolution agent workflow using Mastra AI stateful workflows and vector memory.

Delivery guide

How work is evaluated

Evaluation

Evaluates grievance resolution accuracy and payout calculation logic.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

policy_clause_reference_check

Verifies that all rulings cite a valid policy clause string.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

resolution_agreement

Agreement rate with human benchmark rulings. • target: 0.9 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Build complex workflows using Mastra AI in TypeScript

  • Utilize Mastra vector memory to recall historical dispute precedents

  • Implement multi-step escalation logic for complex customer-partner disputes

  • Validate worker compensation adjustments against platform terms of service

Resources and assets

Reference links and supporting material

Dataset notes

150 gig worker dispute appeal transcripts and gold benchmark resolutions.

Challenge setup

Evaluates grievance resolution accuracy and payout calculation logic.

Challenge input

JSON containing ticket_id, partner_id, dispute_type, worker_statement, and customer_rating.

Mastra AI

Offers stateful TS workflows and built-in memory management.

Evaluated output

JSON containing resolution ('FAVOR_WORKER', 'FAVOR_CUSTOMER', 'NO_ACTION'), payout_compensation, and policy_clause.

Evaluation checks
  • Verifies that all rulings cite a valid policy clause string.
  • Agreement rate with human benchmark rulings. • target: 0.9 • range: 0-1
Configured evaluation evidence
  • Benchmark: Grievance Arbitrator Evaluation Benchmark
  • Resolution Agreement target: 0.9
  • 1 public reference case
Configured runtime
  • JavaScript execution harness
  • javascript sandbox (unavailable on Versalist)

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Mastra AI · Required
  • RAI · Optional
  • Ellipsis · Optional
Orchestration
  • Mastra AI · Required

Evaluation contract

  • policy_clause_reference_check · Weight 1
  • resolution_agreement · Weight 1

Metrics

  • resolution_agreement · Target 0.9

Benchmarks

  • Grievance Arbitrator Evaluation Benchmark
  • general

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON containing ticket_id, partner_id, dispute_type, worker_statement, and customer_rating.

    Output: JSON containing resolution ('FAVOR_WORKER', 'FAVOR_CUSTOMER', 'NO_ACTION'), payout_compensation, and policy_clause.

    Execution harness: JavaScript

Evaluation tools

  • Mastra AI · Workflow Engine

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Gig Worker Grievance Redressal Workflow using Mastra AI