Child Safety in AI Chatbots
Given the FTC's investigation into the impact of AI chatbots on children, create a system for detecting and mitigating risks to children interacting with AI chatbots. This system should include a model that identifies potentially harmful content (e.g., inappropriate language, harmful suggestions) and a mechanism to flag or block such content. Consider ethical implications and potential biases in your design.
What you are building
The core problem, expected build, and operating context for this challenge.
Given the FTC's investigation into the impact of AI chatbots on children, create a system for detecting and mitigating risks to children interacting with AI chatbots. This system should include a model that identifies potentially harmful content (e.g., inappropriate language, harmful suggestions) and a mechanism to flag or block such content. Consider ethical implications and potential biases in your design.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
What you should walk away with
Learning objectives will be added soon
Use the overview and evaluation guide as the source of truth for expected outcomes.
How this agent runs
The system will be evaluated on its ability to accurately detect harmful content, its effectiveness in mitigating risks, and its consideration of ethical implications. Testing will include a combination of automated a...
Challenge input
Text input from a chatbot conversation.
Agent execution
The configured agent processes the input under the challenge policy.
Evaluated output
JSON with a risk score and a classification (harmful/safe).
- The evaluator checks the declared output contract.
- Accuracy target: 0.85
- Python execution harness
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- The evaluation module defines the checks.
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset and AI stack
Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.
Scope a managed run[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.