Building Adversarial Deepfake Detection
Following reports of advanced AI models generating non-consensual deepfakes, this challenge focuses on developing an ethical AI solution for robust deepfake detection and content moderation. You will build a multi-agent system using AutoGen, where agents collaborate and debate to identify and flag AI-generated visual content that violates ethical guidelines. The system will integrate OpenAI 'Next-Gen Multimodal' capabilities for nuanced image and text analysis, enabling the agents to understand context and detect subtle manipulations. A2A Protocol will facilitate secure and structured communication between 'Moderation' agents and 'Red Team' agents, with the latter attempting to bypass the moderation system with adversarial prompts, pushing the boundaries of ethical AI robustness. Extended thinking and dynamic policy RAG will ensure comprehensive and adaptable moderation.
What you are building
The core problem, expected build, and operating context for this challenge.
Multi-agent system leveraging AutoGen, OpenAI 'Next-Gen Multimodal', and A2A Protocol for ethical content moderation and adversarial deepfake detection.
How work is evaluated
The system will be evaluated on its accuracy in detecting various deepfake types, adherence to moderation policies, and robustness against adversarial prompts designed to bypass its defenses.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Build adversarial robustness mechanisms against prompt engineering attacks.
Design and enforce dynamic ethical content moderation policies.
Develop a multi-agent system for deepfake detection using AutoGen.
Integrate OpenAI 'Next-Gen Multimodal' for image and text analysis.
Implement A2A Protocol for secure agent-to-agent collaboration and red-teaming.
Reference links and supporting material
Sample data for 1 tasks
Challenge setup
The system will be evaluated on its accuracy in detecting various deepfake types, adherence to moderation policies, and robustness against adversarial prompts designed to bypass its defenses.
Challenge input
{ 'scenario_id': 'string', 'items': [{ 'item_id': 'string', 'image_url': 'string', 'prompt_text': 'string', 'is_deepfake': 'boolean', 'is_adversari...
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{ 'scenario_id': 'string', 'results': [{ 'item_id': 'string', 'detected_as_deepfake': 'boolean', 'moderation_action_taken': 'string', 'reasoning':...
- Successfully identifies deepfakes (true positives) and avoids misclassifying real content (true negatives).
- Takes the correct moderation action based on the expected policy for each item.
- Agent system successfully flags adversarial attempts to generate deepfakes.
- TruePositiveRate target: 90%
- Python execution harness
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Successfully identifies deepfakes (true positives) and avoids misclassifying real content (true negatives).
- Takes the correct moderation action based on the expected policy for each item.
- Agent system successfully flags adversarial attempts to generate deepfakes.
Metrics
- TruePositiveRate · Target 0.9
- FalsePositiveRate · Target 0.05
- RedTeamBypassRate · Target 0.05
Workflow tasks
- Task 1
Input: { 'scenario_id': 'string', 'items': [{ 'item_id': 'string', 'image_url': 'string', 'prompt_text': 'string', 'is_deepfake': 'boolean', 'is_adversarial': 'boolean', 'expected_moderation_action': 'string' }] }
Output: { 'scenario_id': 'string', 'results': [{ 'item_id': 'string', 'detected_as_deepfake': 'boolean', 'moderation_action_taken': 'string', 'reasoning': 'string', 'agent_debate_summary': 'string' }], 'overall_confidence': 'float' }
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.