Back to challenges
Challenge

AutoGen & Braintrust Trade Finance Audit Agent

Build an AutoGen multi-agent system evaluated via Braintrust to automate trade finance document review, sanction checking, and anomaly detection across cross-border Letter of Credit applications. Achieve 95%+ precision on flagging regulatory discrepancies against Monetary Authority of Singapore guidelines while preserving full audit trails.

Workflow AutomationHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop a multi-agent orchestration workflow using AutoGen and Braintrust to evaluate cross-border LC documents against Singapore trade finance rules.

Delivery guide

How work is evaluated

Evaluation

Evaluates AutoGen multi-agent accuracy and audit trail completeness against trade finance datasets via Braintrust metrics.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Valid Decision Output

Checks if the decision is APPROVED, REJECTED, or ESCALATED

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

compliance_accuracy

Accuracy of trade violation detection • target: 0.95 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master multi-agent orchestration with AutoGen for complex financial documents

  • Implement continuous automated evaluation using Braintrust

  • Enforce compliance and regulatory rules for Singapore trade finance

  • Build human-in-the-loop exception handling for high-risk flags

Resources and assets

Reference links and supporting material

Dataset notes

Synthetic cross-border Letter of Credit dataset with annotated compliance defects and sanctioned entity flags based on MAS guidelines.

How this agent runs

Evaluates AutoGen multi-agent accuracy and audit trail completeness against trade finance datasets via Braintrust metrics.

Challenge input

JSON containing trade finance document text and metadata

AutoGen

Multi-agent conversational workflows for trading document analysis

Braintrust

Enterprise evaluation, tracing, and metric scoring

Evaluated output

JSON containing risk score, list of compliance violations, and agent decision trail

Checks for
  • Checks if the decision is APPROVED, REJECTED, or ESCALATED
  • Accuracy of trade violation detection • target: 0.95 • range: 0-1
Proof of success
  • Benchmark: FinQA / TradeFinanceBench
  • Compliance Accuracy target: 0.95
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Tabnine · Optional
Observation
  • Braintrust · Optional
Reward / Eval
  • Braintrust · Optional

Evaluation contract

  • Valid Decision Output · Weight 1
  • compliance_accuracy · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about AutoGen & Braintrust Trade Finance Audit Agent