Skip to main content
Back to challenges
Challenge

Agents for Prompt-Driven Brand Sentiment & Affinity

This challenge focuses on building a sophisticated multi-agent system using CrewAI to analyze brand mentions and sentiment within a stream of AI-generated prompts. The system will orchestrate specialized agents to monitor, categorize, and report on brand affinity. The core idea is to simulate an intelligent monitoring platform that provides actionable insights into brand perception and recommendation patterns from diverse data sources, leveraging CrewAI's role-playing architecture. Developers will design a collaborative agent team, where each agent has a distinct role, tools, and goals. For instance, a 'Prompt Ingestor' agent, a 'Brand Analyst' agent, and a 'Report Generator' agent will work in concert. The system will use DeepSeek R1 for its advanced reasoning capabilities to accurately interpret nuanced sentiment and complex brand associations. Integration with Zapier Interfaces will enable seamless data ingestion from various sources, while tool can facilitate custom workflow automation for alert triggers and data processing. Voiceflow will provide a conversational interface for real-time query and status updates, making the system highly interactive. Blaxel will serve as the underlying agent orchestration backbone, ensuring robust agent management.

Special Purpose AgentsHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a CrewAI-based multi-agent system leveraging DeepSeek R1 for analyzing brand sentiment and affinity from prompts, integrated with workflow automation and conversational interfaces.

Delivery guide

How work is evaluated

Evaluation

The evaluation will assess the system's ability to accurately identify brand mentions, analyze sentiment, and generate coherent reports based on simulated prompt data. It will also test the functionality of tool integrations and the Voiceflow interface.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
6 scoring checks
Binary
6 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

SentimentAccuracy

At least 85% accuracy in identifying sentiment for given brand mentions.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

BrandExtractionCompleteness

Identifies all relevant brand mentions (80% recall).

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 3

KoreAiIntegration

Kore.ai workflow is successfully triggered on specified conditions.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 4

VoiceflowResponsiveness

Voiceflow agent provides coherent and relevant responses to queries.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 5

AverageSentimentAccuracy

Average accuracy of sentiment classification across all extracted brands. • target: 0.9 • range: 0.75-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 6

ToolUsageEffectiveness

Percentage of tasks where tools were correctly invoked and utilized. • target: 0.95 • range: 0.8-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Master CrewAI for building robust, role-based agent teams with defined responsibilities and collaborative workflows for data analysis.

  • Implement advanced sentiment analysis and entity extraction using DeepSeek R1's API for nuanced understanding of brand mentions in diverse prompts.

  • Design and build custom tools for CrewAI agents to programmatically interact with Zapier Interfaces for data ingestion and event triggering.

  • Integrate Kore.ai to define and execute complex workflow automations based on agent-detected insights and threshold breaches.

  • Develop a Voiceflow project to create an intuitive voice-enabled or chat-based interface for querying system status and brand analytics.

  • Leverage Blaxel's capabilities for monitoring agent interactions, managing agent lifecycles, and ensuring system resilience in production environments.

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 3 tasks

Challenge setup

The evaluation will assess the system's ability to accurately identify brand mentions, analyze sentiment, and generate coherent reports based on simulated prompt data. It will also test the functionality of tool integ...

Challenge input

JSON array of strings, each string representing an AI-generated prompt.

CrewAI

Framework for orchestrating

Zed

High-performance code editor

DeepSeek R1

Policy Serving in the agent workflow.

Evaluated output

JSON object with brand names as keys, each containing an array of identified sentiments (e.g., 'positive', 'negative', 'neutral') and associated pr...

Evaluation checks
  • At least 85% accuracy in identifying sentiment for given brand mentions.
  • Identifies all relevant brand mentions (80% recall).
  • Kore.ai workflow is successfully triggered on specified conditions.
Configured evaluation evidence
  • AverageSentimentAccuracy target: 0.9
  • 3 public reference cases
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • CrewAI · Required
  • Zed · Optional
Policy Serving
  • DeepSeek R1 · Optional

Evaluation contract

  • SentimentAccuracy · Weight 1
  • BrandExtractionCompleteness · Weight 1
  • KoreAiIntegration · Weight 1
  • VoiceflowResponsiveness · Weight 1
  • AverageSentimentAccuracy · Weight 1
  • ToolUsageEffectiveness · Weight 1

Metrics

  • AverageSentimentAccuracy · Target 0.9
  • ToolUsageEffectiveness · Target 0.95

Reference cases

  • 3 public reference cases

Workflow tasks

  1. Task 1

    Input: JSON array of strings, each string representing an AI-generated prompt.

    Output: JSON object with brand names as keys, each containing an array of identified sentiments (e.g., 'positive', 'negative', 'neutral') and associated prompt snippets.

    Execution harness: Python

  2. Task 2

    Input: Simulated event data indicating a negative sentiment threshold breach for a brand.

    Output: Confirmation of Kore.ai workflow execution (e.g., webhook log entry, specific API call made).

    Execution harness: Python

  3. Task 3

    Input: String representing a natural language query (e.g., 'What's the current sentiment for Product X?').

    Output: String response from the Voiceflow agent, containing relevant sentiment data or status.

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Agents for Prompt-Driven Brand Sentiment & Affinity