Build an MLflow Accelerator Procurement Scenario Tracker
Compare cloud accelerator reservation options against generative inference demand curves. Track multi-scenario procurement parameters, break-even metrics, and API-generated cost narratives using MLflow to manage financial lifecycle evaluations.
What you are building
The core problem, expected build, and operating context for this challenge.
Compare cloud accelerator reservation options against generative inference demand curves. Track multi-scenario procurement parameters, break-even metrics, and API-generated cost narratives using MLflow to manage financial lifecycle evaluations.
AI FinOps Analyst
Parse projected token request distributions and burst patterns from normalized trace files using pandas.
Calculate baseline costs and break-even hours across on-demand and reserved accelerator pricing scenarios based on projected and billed hours.
Use the Anthropic API to synthesize qualitative cost narratives covering hardware obsolescence and capacity planning tradeoffs based on the calculated metrics.
Log scenario parameters, financial metrics, and the generated comparative procurement narrative into an MLflow tracking server as experiment runs.
Multi-scenario financial comparison workbench with downloadable MLflow metrics and text summaries.
Simplified synthetic cloud accelerator pricing catalogs and normalized traffic traces in a test folder. Starter examples below are synthetic. They are not downloaded records.
AWS Neuron documentation Models AWS EC2 accelerated instance types (e.g., Inf2, Trn1) across commitment tiers.
mlflow, anthropic, pandas. MLflow tracks procurement simulation runs, cost parameters, and comparative scenario metrics. Anthropic API access for generating qualitative procurement sensitivity evaluations.
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.
Evaluates total spend differences between on-demand and reserved instances accurately.
Calculates the exact monthly break-even hours for a reserved instance given on-demand and reserved rates.
Validates pricing schemas against standardized multi-cloud rate formats.
Emits formal procurement brief tracking total cost metrics into an MLflow experiment run.
```json [ { "input": { "accelerator_type": "AWS_INF2", "hourly_on_demand_usd": 10, "hourly_reserved_usd": 6, "projected_monthly_hours": 600, "workload_months": 12, "reservation_billed_hours_per_month": 730 }, "expected_output": { "total_on_demand_cost": 72000, "total_reserved_cost": 52560, "projected_savings_usd": 19440, "break_even_runtime_hours_per_month": 438, "procurement_recommendation": "COMMIT_RESERVED" } }, { "input": { "accelerator_type": "AWS_TRN1", "hourly_on_demand_usd": 20, "hourly_reserved_usd": 12, "projected_monthly_hours": 150, "workload_months": 6, "reservation_billed_hours_per_month": 730 }, "expected_output": { "total_on_demand_cost": 18000, "total_reserved_cost": 52560, "projected_savings_usd": -34560, "break_even_runtime_hours_per_month": 438, "procurement_recommendation": "RETAIN_ON_DEMAND" } } ] ```
Procurement comparisons are simulation only. No automated purchasing or cloud account binding. Explicitly label normalized examples as simplified synthetic app records, not complete official schema files. Use mock external writes. Require human review before any real deployment.
Participation guide
Procurement comparisons are simulation only. No automated purchasing or cloud account binding. Explicitly label normalized examples as simplified synthetic app records, not complete official schema files. Use mock external writes and human review.
How work is evaluated
Evaluates total spend differences between on-demand and reserved instances accurately. Calculates the exact monthly break-even hours for a reserved instance given on-demand and reserved rates. Validates pricing schemas against standardized multi-cloud rate formats. Emits formal procurement brief tracking total cost metrics into an MLflow experiment run.
Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.
Challenge datasets
Public datasets, plus the private datasets you have access to.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
Complete workflow
Parse projected token request distributions and burst patterns from normalized trace files using pandas. Calculate baseline costs and break-even hours across on-demand and reserved accelerator pricing scenarios based on projected and billed hours. Use the Anthropic API to synthesize qualitative cost narratives covering hardware obsolescence and capacity planning tradeoffs based on the calculated metrics. Log scenario parameters, financial metrics, and the generated comparative procurement narrative into an MLflow tracking server as experiment runs.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Correctness and evidence
Evaluates total spend differences between on-demand and reserved instances accurately. Calculates the exact monthly break-even hours for a reserved instance given on-demand and reserved rates. Validates pricing schemas against standardized multi-cloud rate formats. Emits formal procurement brief tracking total cost metrics into an MLflow experiment run.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Review interface
Multi-scenario financial comparison workbench with downloadable MLflow metrics and text summaries.
Missing or incorrect.
Partly complete, with documented gaps.
Complete, with reproducible evidence.
Safe execution
Procurement comparisons are simulation only. No automated purchasing or cloud account binding. Explicitly label normalized examples as simplified synthetic app records, not complete official schema files.
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Parse projected token request distributions and burst patterns from normalized trace files using pandas.
Calculate baseline costs and break-even hours across on-demand and reserved accelerator pricing scenarios based on projected and billed hours.
Use the Anthropic API to synthesize qualitative cost narratives covering hardware obsolescence and capacity planning tradeoffs based on the calculated metrics.
Log scenario parameters, financial metrics, and the generated comparative procurement narrative into an MLflow tracking server as experiment runs.
Reference links and supporting material
Simplified synthetic cloud accelerator pricing catalogs and normalized traffic traces in a test folder. Supplied starter examples are synthetic, not source extracts.
Challenge setup
Configuration summary for this challenge.
Challenge input
Input details are not configured.
MLflow
MLOps
Evaluated output
No output checks are configured.
- Parse projected token request distributions and burst patterns from normalized trace files using pandas. Ca...
- Evaluates total spend differences between on-demand and reserved instances accurately. Calculates the exact...
- Multi-scenario financial comparison workbench with downloadable MLflow metrics and text summaries.
- 2 public reference cases
- 1 configured tool
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- MLflow · Optional
Evaluation contract
- Complete workflow · Weight 40
- Correctness and evidence · Weight 30
- Review interface · Weight 20
- Safe execution · Weight 10
Reference cases
- 2 public reference cases
Workflow tasks
No workflow tasks are configured.
Recipe freeze marker
No recipe freeze time is recorded.