Back to evaluations
Public evaluation

TranslationAccuracy

Evaluation will focus on the accuracy of multilingual translation, the precision and recall of structured information extraction, the robustness of the ML pipeline, and the usability and responsiveness of the interactive chat assistant.

Evaluation type
task based
Challenge
Multilingual Policy Analyzer via Fine-Tuned TranslateGemma
Difficulty
Intermediate
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
2
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

TranslationAccuracy

Evaluates the quality of translation for policy documents from source to target languages using TranslateGemma.

Input format

{ "document_text": "string", "source_language": "string", "target_language": "string" }

Output format

{ "translated_text": "string" }

Task 2

InformationExtractionPrecision

Assesses the accuracy of extracting specific entities and commitments from translated policy documents.

Input format

{ "translated_document": "string", "extraction_schema": { "entity_type": "string" } }

Output format

{ "extracted_data": [{"entity_type": "string", "value": "string", "confidence": "float"}] }