Building Adversarial Deepfake Detection
Following reports of advanced AI models generating non-consensual deepfakes, this challenge focuses on developing an ethical AI solution for robust deepfake detection and content moderation. You will build a multi-agent system using AutoGen, where agents collaborate and debate to identify and flag AI-generated visual content that violates ethical guidelines. The system will integrate OpenAI 'Next-Gen Multimodal' capabilities for nuanced image and text analysis, enabling the agents to understand context and detect subtle manipulations. A2A Protocol will facilitate secure and structured communication between 'Moderation' agents and 'Red Team' agents, with the latter attempting to bypass the moderation system with adversarial prompts, pushing the boundaries of ethical AI robustness. Extended thinking and dynamic policy RAG will ensure comprehensive and adaptable moderation.
What you are building
Multi-agent system leveraging AutoGen, OpenAI 'Next-Gen Multimodal', and A2A Protocol for ethical content moderation and adversarial deepfake detection.
How work is evaluated
The system will be evaluated on its accuracy in detecting various deepfake types, adherence to moderation policies, and robustness against adversarial prompts designed to bypass its defenses.
Datasets
Public datasets, plus the private datasets you have access to.
Learning goals
Build adversarial robustness mechanisms against prompt engineering attacks.
Design and enforce dynamic ethical content moderation policies.
Develop a multi-agent system for deepfake detection using AutoGen.
Integrate OpenAI 'Next-Gen Multimodal' for image and text analysis.
Implement A2A Protocol for secure agent-to-agent collaboration and red-teaming.
Resources
Sample data for 1 tasks
Challenge setup
The system will be evaluated on its accuracy in detecting various deepfake types, adherence to moderation policies, and robustness against adversarial prompts designed to bypass its defenses.
Challenge input
{ 'scenario_id': 'string', 'items': [{ 'item_id': 'string', 'image_url': 'string', 'prompt_text': 'string', 'is_deepfake': 'boolean', 'is_adversari...
Agent execution
No agent tool is listed for this challenge.
Evaluated output
{ 'scenario_id': 'string', 'results': [{ 'item_id': 'string', 'detected_as_deepfake': 'boolean', 'moderation_action_taken': 'string', 'reasoning':...
- Successfully identifies deepfakes (true positives) and avoids misclassifying real content (true negatives).
- Takes the correct moderation action based on the expected policy for each item.
- Agent system successfully flags adversarial attempts to generate deepfakes.
- TruePositiveRate target: 90%
- Python execution harness
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- Successfully identifies deepfakes (true positives) and avoids misclassifying real content (true negatives).
- Takes the correct moderation action based on the expected policy for each item.
- Agent system successfully flags adversarial attempts to generate deepfakes.
Metrics
- TruePositiveRate · Target 0.9
- FalsePositiveRate · Target 0.05
- RedTeamBypassRate · Target 0.05
Workflow tasks
- Task 1
Input: { 'scenario_id': 'string', 'items': [{ 'item_id': 'string', 'image_url': 'string', 'prompt_text': 'string', 'is_deepfake': 'boolean', 'is_adversarial': 'boolean', 'expected_moderation_action': 'string' }] }
Output: { 'scenario_id': 'string', 'results': [{ 'item_id': 'string', 'detected_as_deepfake': 'boolean', 'moderation_action_taken': 'string', 'reasoning': 'string', 'agent_debate_summary': 'string' }], 'overall_confidence': 'float' }
Execution harness: Python
Recipe freeze marker
No recipe freeze time is recorded.