AutoGen & Braintrust Trade Finance Audit Agent
Build an AutoGen multi-agent system evaluated via Braintrust to automate trade finance document review, sanction checking, and anomaly detection across cross-border Letter of Credit applications. Achieve 95%+ precision on flagging regulatory discrepancies against Monetary Authority of Singapore guidelines while preserving full audit trails.
What you are building
The core problem, expected build, and operating context for this challenge.
Develop a multi-agent orchestration workflow using AutoGen and Braintrust to evaluate cross-border LC documents against Singapore trade finance rules.
How work is evaluated
Evaluates AutoGen multi-agent accuracy and audit trail completeness against trade finance datasets via Braintrust metrics.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Valid Decision Output
Checks if the decision is APPROVED, REJECTED, or ESCALATED
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
compliance_accuracy
Accuracy of trade violation detection • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master multi-agent orchestration with AutoGen for complex financial documents
Implement continuous automated evaluation using Braintrust
Enforce compliance and regulatory rules for Singapore trade finance
Build human-in-the-loop exception handling for high-risk flags
Reference links and supporting material
Synthetic cross-border Letter of Credit dataset with annotated compliance defects and sanctioned entity flags based on MAS guidelines.
Challenge setup
Evaluates AutoGen multi-agent accuracy and audit trail completeness against trade finance datasets via Braintrust metrics.
Challenge input
JSON containing trade finance document text and metadata
AutoGen
Multi-agent conversational workflows for trading document analysis
Braintrust
Enterprise evaluation, tracing, and metric scoring
Evaluated output
JSON containing risk score, list of compliance violations, and agent decision trail
- Checks if the decision is APPROVED, REJECTED, or ESCALATED
- Accuracy of trade violation detection • target: 0.95 • range: 0-1
- Benchmark: FinQA / TradeFinanceBench
- Compliance Accuracy target: 0.95
- 1 public reference case
- Python execution harness
- Python sandbox (unavailable on Versalist)
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- AutoGen · Required
- Tabnine · Optional
- Braintrust · Optional
- Braintrust · Optional
Evaluation contract
- Valid Decision Output · Weight 1
- compliance_accuracy · Weight 1
Metrics
- compliance_accuracy · Target 0.95
Benchmarks
- FinQA / TradeFinanceBench
- security
Reference cases
- 1 public reference case
Workflow tasks
- Task 1
Input: JSON containing trade finance document text and metadata
Output: JSON containing risk score, list of compliance violations, and agent decision trail
Execution harness: Python
Evaluation tools
- AutoGen · Agent Orchestration
- Braintrust · AI Workflow Automation
Recipe freeze marker
No recipe freeze time is recorded.