Back to challenges
Challenge

Type-Safe Token and Power Attribution Engine with Pydantic AI

Attribute hardware power consumption and financial cost directly to individual tenant API requests. Build a type-safe token telemetry and energy accounting service using Pydantic AI that parses Prometheus hardware metrics and maps Watts and execution time back to tenant request contexts.

EngineeringHosted by Vera
Sign in free to start
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Build a type-safe energy attribution engine with Pydantic AI to validate, parse, and assign server Watt consumption to specific tenant token generation requests.

Delivery guide

How work is evaluated

Evaluation

Evaluates Pydantic schema validation accuracy and energy attribution calculations.

Datasets

Shared data for this challenge

Review public datasets and any private uploads tied to your build.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Pydantic Schema Validation

Ensure output strictly conforms to Pydantic model structure.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Attribution Accuracy

Precision of calculated energy consumption vs energy formula. • target: 1 • range: 0-1

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Define strict Pydantic schemas for multi-tenant power and token usage payloads.

  • Build a Pydantic AI agent agentic pipeline to extract power draw from Prometheus time-series metrics.

  • Accurately calculate micro-dollar and Joules-per-token metrics per query.

Resources and assets

Reference links and supporting material

Dataset notes

Prometheus energy metrics dataset containing GPU Watt measurements and tenant request timestamps.

How this agent runs

Evaluates Pydantic schema validation accuracy and energy attribution calculations.

Challenge input

JSON tenant_id, execution_time_s, avg_power_draw_watts, energy_cost_per_kwh

Pydantic AI

Type-safe Python agent framework built on Pydantic.

Evaluated output

JSON valid Pydantic model containing energy_consumed_joules, financial_cost_usd

Checks for
  • Ensure output strictly conforms to Pydantic model structure.
  • Precision of calculated energy consumption vs energy formula. • target: 1 • range: 0-1
Proof of success
  • Benchmark: Green AI Hardware Telemetry Standard
  • Attribution Accuracy target: 1
  • 1 public reference case
Runtime evidence
  • Python execution harness
  • Python sandbox (unavailable on Versalist)
View technical recipe

Configured tools

Action Space
  • Pydantic AI · Required
  • Anthropic · Optional
  • Anthropic Claude · Optional
Orchestration
  • Pydantic AI · Required

Evaluation contract

  • Pydantic Schema Validation · Weight 1
  • Attribution Accuracy · Weight 1

Recipe state

This is a preview. The configuration can change before the evaluation recipe is locked.

Frequently Asked Questions about Type-Safe Token and Power Attribution Engine with Pydantic AI