Back to evaluations
Draft evaluation

Extract Short-Term Rental Analytics with Pydantic AI and Together AI — evaluation

Evaluates Pydantic schema validation success and RevPAR metric extraction accuracy.

Evaluation type
task based
Challenge
Extract Short-Term Rental Analytics with Pydantic AI and Together AI
Difficulty
Intermediate
Rigor
Not declared

The author has not specified a rigor level.

Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
1
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

Vacation Rental Market Extraction

Ingests raw text rental reports and outputs validated Pydantic metric model.

Input format

JSON with market_name, raw_report_text

Output format

JSON conforming to VacationMarketStats Pydantic model