Skip to main content
Challenge

Build an AWS Neuron Spot Preemption Profiler

Simulate EC2 Inf2 and Trn1 accelerator spot preemption risks against open-source transformer batch workloads. Quantify interruption buffers and synthesize preemption tolerance reports.

EngineeringHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Simulate EC2 Inf2 and Trn1 accelerator spot preemption risks against open-source transformer batch workloads. Quantify interruption buffers and synthesize preemption tolerance reports.

Cloud Operations Procurement Specialist

Ingest workload batch concurrency requirements and historical spot availability SLA metrics.

Evaluate AWS Neuron EC2 instance types including inf2.xlarge and trn1.32xlarge spot interruption profiles.

Simulate batch failure rates and interruption risk buffers during peak serving hours.

Use hosted Claude via Anthropic API to synthesize an executive spot viability proposal.

Web scenario workbench with interactive slider controls for spot tolerance and reservation terms.

Construct a synthetic tabular catalog of AWS EC2 Neuron instance rates and monthly batch inference traffic profiles. Starter examples below are synthetic. They are not downloaded records.

AWS Neuron documentation Reference AWS Neuron accelerator families including Inferentia and Trainium instances (Inf2, Trn1).

anthropic, pydantic, pandas. anthropic-claude compiles scenario modeling metrics into executive business justifications and trade-off tables. Hosted Claude API for summarizing financial scenario models and drafting procurement recommendations.

Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

Calculates batch failure rates based on historical spot availability SLAs.

Models interruption risk buffers for spot instances during peak batch serving hours.

Outputs a structured viability recommendation matrix with interruption tolerance thresholds.

```json [ { "input": { "monthly_hours_needed": 720, "instance_family": "trn1.32xlarge", "spot_availability_sla": 0.85, "batch_tolerance_sla": 0.95 }, "expected_output": { "simulated_failure_rate": 0.15, "buffer_needed": 1.18, "recommended_strategy": "unsuitable_due_to_preemption_risk", "spot_viability": "FAIL" } }, { "input": { "monthly_hours_needed": 200, "instance_family": "inf2.8xlarge", "spot_availability_sla": 0.95, "batch_tolerance_sla": 0.9 }, "expected_output": { "simulated_failure_rate": 0.05, "buffer_needed": 1.05, "recommended_strategy": "spot_with_fallback", "spot_viability": "PASS" } } ] ```

Performs financial simulation only; does not execute binding EC2 reservation purchases or cloud API purchases. Use mock external writes. Require human review before any real deployment.

Rules

Participation guide

Performs financial simulation only; does not execute binding EC2 reservation purchases or cloud API purchases. Use mock external writes and human review.

Delivery guide

How work is evaluated

Evaluation

Calculates batch failure rates based on historical spot availability SLAs. Models interruption risk buffers for spot instances during peak batch serving hours. Outputs a structured viability recommendation matrix with interruption tolerance thresholds.

Submission format

Runnable source, pinned dependencies, setup commands, working interface, example outputs, recorded demo, tests, and action logs.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
4 scoring checks
Binary
1 pass or fail dimensions
Ordinal
3 scaled dimensions
Dimension 1

Complete workflow

Ingest workload batch concurrency requirements and historical spot availability SLA metrics. Evaluate AWS Neuron EC2 instance types including inf2.xlarge and trn1.32xlarge spot interruption profiles. Simulate batch failure rates and interruption risk buffers during peak serving hours. Use hosted Claude via Anthropic API to synthesize an executive spot viability proposal.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 2

Correctness and evidence

Calculates batch failure rates based on historical spot availability SLAs. Models interruption risk buffers for spot instances during peak batch serving hours. Outputs a structured viability recommendation matrix with interruption tolerance thresholds.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 3

Review interface

Web scenario workbench with interactive slider controls for spot tolerance and reservation terms.

Score scale
012
Score 0

Missing or incorrect.

Score 1

Partly complete, with documented gaps.

Score 2

Complete, with reproducible evidence.

Dimension 4

Safe execution

Performs financial simulation only; does not execute binding EC2 reservation purchases or cloud API purchases.

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Ingest workload batch concurrency requirements and historical spot availability SLA metrics.

  • Evaluate AWS Neuron EC2 instance types including inf2.xlarge and trn1.32xlarge spot interruption profiles.

  • Simulate batch failure rates and interruption risk buffers during peak serving hours.

  • Use hosted Claude via Anthropic API to synthesize an executive spot viability proposal.

Resources and assets

Reference links and supporting material

Dataset notes

Construct a synthetic tabular catalog of AWS EC2 Neuron instance rates and monthly batch inference traffic profiles. Supplied starter examples are synthetic, not source extracts.

Challenge setup

Configuration summary for this challenge.

Challenge input

Input details are not configured.

Anthropic Claude

LLM for code generation & more

Evaluated output

No output checks are configured.

Evaluation checks
  • Ingest workload batch concurrency requirements and historical spot availability SLA metrics. Evaluate AWS N...
  • Calculates batch failure rates based on historical spot availability SLAs. Models interruption risk buffers...
  • Web scenario workbench with interactive slider controls for spot tolerance and reservation terms.
Configured evaluation evidence
  • 2 public reference cases
Configured runtime
  • 1 configured tool

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Anthropic Claude · Optional

Evaluation contract

  • Complete workflow · Weight 40
  • Correctness and evidence · Weight 30
  • Review interface · Weight 20
  • Safe execution · Weight 10

Reference cases

  • 2 public reference cases

Workflow tasks

No workflow tasks are configured.

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Build an AWS Neuron Spot Preemption Profiler