Dialogue Generation and Character Consistency
The evaluation will assess the Character Persona Engine's ability to generate lore-consistent dialogue, accurately recall character facts from the knowledge graph, and maintain character voice across diverse scenarios.
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Task templates
Inputs and expected outputs.
Dialogue Generation and Character Consistency
Given a character, a scenario, and a dialogue prompt, the engine must generate a response that is consistent with the character's established personality, motivations, and lore.
{"character_name": "string", "scenario_description": "string", "user_prompt": "string"}
{"generated_dialogue": "string", "lore_references": ["string", ...]}
Lore Recall Accuracy (Graph RAG)
The engine must accurately retrieve specific, non-obvious facts about a character from its knowledge graph and integrate them into a coherent answer.
{"character_name": "string", "question": "string"}
{"answer": "string", "retrieved_graph_nodes": ["string", ...]}
MCP Tool Integration for Lore Update
Demonstrate the ability to invoke a mock MCP-enabled tool to 'update' or 'add' a new lore fact to the character's knowledge graph.
{"tool_invocation_request": {"tool_name": "string", "parameters": {"character_name": "string", "fact_type": "string", "fact_value": "string"}}}
{"tool_response": "string", "success": "boolean"}