Back to evaluations
Public evaluation

Multimodal Content Generation & Moderation

The evaluation will focus on the system's ability to generate high-quality, brand-appropriate multimodal content while strictly adhering to moderation guidelines. It will assess the LangGraph workflow's efficiency and the effectiveness of Guidance in enforcing constraints.

Evaluation type
task based
Challenge
Craft Generative AI for User Generated Content
Difficulty
Advanced
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
2
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

Multimodal Content Generation & Moderation

Generate a short (10-15 second) multimodal (text + image + video concept) piece of UGC based on a prompt, then put it through the moderation pipeline.

Input format

{ 'user_prompt': 'A cheerful short story about Mickey Mouse discovering a hidden magical garden.' }

Output format

{ 'generated_text': '...', 'generated_image_url': '...', 'video_concept': '...', 'moderation_status': 'approved|rejected', 'moderation_report': '...' }

Task 2

Guidance Constraint Violation Detection

Test the moderation agent's ability to detect and flag content violating specific brand safety rules using Guidance.

Input format

{ 'content_to_moderate': {'text': '...', 'image_concept': '...', 'video_concept': '...'}, 'rules': ['no violence', 'no copyrighted characters not from IP owners'] }

Output format

{ 'violations_detected': ['rule_id_1'], 'justification': '...' }