Skip to main content
Loading navigation
Back to challenges

Type-Safe Token and Power Attribution Engine with Pydantic AI

Attribute hardware power consumption and financial cost directly to individual tenant API requests. Build a type-safe token telemetry and energy accounting service using Pydantic AI that parses Prometheus hardware metrics and maps Watts and execution time back to tenant request contexts.

EngineeringHosted by Vera
Sign in free to start

What you are building

Build a type-safe energy attribution engine with Pydantic AI to validate, parse, and assign server Watt consumption to specific tenant token generation requests.

How work is evaluated

Evaluation

Evaluates Pydantic schema validation accuracy and energy attribution calculations.

Datasets

Public datasets, plus the private datasets you have access to.

How submissions are scored

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Pydantic Schema Validation

Ensure output strictly conforms to Pydantic model structure.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Attribution Accuracy

Precision of calculated energy consumption vs energy formula. • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

  • Define strict Pydantic schemas for multi-tenant power and token usage payloads.

  • Build a Pydantic AI agent agentic pipeline to extract power draw from Prometheus time-series metrics.

  • Accurately calculate micro-dollar and Joules-per-token metrics per query.

Resources

Dataset notes

Prometheus energy metrics dataset containing GPU Watt measurements and tenant request timestamps.

Challenge setup

Evaluates Pydantic schema validation accuracy and energy attribution calculations.

Challenge input

JSON tenant_id, execution_time_s, avg_power_draw_watts, energy_cost_per_kwh

Pydantic AI

Type-safe Python agent framework built on Pydantic.

Evaluated output

JSON valid Pydantic model containing energy_consumed_joules, financial_cost_usd

Evaluation checks
  • Ensure output strictly conforms to Pydantic model structure.
  • Precision of calculated energy consumption vs energy formula. • target: 1 • range: 0-1
Configured evaluation evidence
  • Benchmark: Green AI Hardware Telemetry Standard
  • Attribution Accuracy target: 1
  • 1 public reference case
Configured runtime
  • Python execution harness
  • Python sandbox (unavailable on Versalist)

Shows the first task, up to three tools, and up to three evidence items.

View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • Anthropic · Optional
  • Anthropic Claude · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • Pydantic Schema Validation · Weight 1
  • Attribution Accuracy · Weight 1

Metrics

  • Attribution Accuracy · Target 1

Benchmarks

  • Green AI Hardware Telemetry Standard
  • energy

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: JSON tenant_id, execution_time_s, avg_power_draw_watts, energy_cost_per_kwh

    Output: JSON valid Pydantic model containing energy_consumed_joules, financial_cost_usd

    Execution harness: Python

Evaluation tools

  • Pydantic AI · Agent Framework

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Type-Safe Token and Power Attribution Engine with Pydantic AI