Skip to main content
Loading navigation
Back to challenges

Build a SEC Regulatory Compliance Auditor with AutoGen and Adept

Proposed SEC regulations on crypto assets require continuous auditing of token transaction logs and compliance disclosures. Construct a multi-agent auditing system using AutoGen and Adept to automatically scan ledger transactions and flag regulatory non-compliance.

CybersecurityHosted by Vera
Sign in free to start

What you are building

Build an AutoGen multi-agent system combined with Adept browser automation to audit crypto exchange regulatory compliance reports.

How work is evaluated

Evaluation

Evaluates multi-agent compliance verification accuracy against sample crypto custody logs.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

test_violation_detection

Verify commingling detected correctly as non-compliant

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

audit_accuracy

Accuracy of regulatory non-compliance detection • target: 0.96 • range: 0.9-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Design an AutoGen multi-agent team (Auditor, Legal Examiner, Reporter).

  • Use Adept actions to navigate regulatory filing portals and verify ledger state.

  • Generate automated SEC disclosure compliance audit reports.

Resources

Dataset notes

Synthetic cryptocurrency exchange transaction records and custody balance snapshots.

Challenge setup

Evaluates multi-agent compliance verification accuracy against sample crypto custody logs.

Challenge input

JSON array of daily custody balances and wallet addresses

AutoGen

Multi-agent framework requirement

Adept

AI workflow automation requirement

Evaluated output

JSON object with compliance_status, violations, and summary

Evaluation checks
  • Verify commingling detected correctly as non-compliant
  • Accuracy of regulatory non-compliance detection • target: 0.96 • range: 0.9-1
Configured evaluation evidence
  • Benchmark: Compliance Agent Benchmark v1
  • Audit Accuracy target: 0.96
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • AutoGen · Required
  • Adept · Optional
  • Microsoft AutoGen · Optional

Evaluation contract

  • test_violation_detection · Weight 1
  • audit_accuracy · Weight 1

Metrics

  • audit_accuracy · Target 0.96

Benchmarks

  • Compliance Agent Benchmark v1
  • security

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON array of daily custody balances and wallet addresses

    Output: JSON object with compliance_status, violations, and summary

    Execution harness: Python

Evaluation tools

  • AutoGen · agent_framework
  • Adept · workflow_automation

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build a SEC Regulatory Compliance Auditor with AutoGen and Adept