Back to evaluations
Public evaluation
Observability Reconciler Test
Evaluates cost and energy attribution accuracy across multi-tenant trace runs.
Evaluation type
task based
Challenge
Hardware Observability Pipeline for Token, Watt, and Cost Attribution with LangChain
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
Observability Reconciler Test
Processes 10,000 token trace logs and verifies tenant cost breakdown reconciliation.
Input format
JSON array of trace events with NVML power samples
Output format
JSON report containing cost_attribution_by_tenant and error_margin_percent