Build the application

## Build
Benchmark draft and target open-weight model pairings under local vLLM serving. Synthesize speedup expectations and memory trade-offs using hosted Claude, outputting validated deployment configuration schemas.

## User
Inference Optimization Specialist

## Workflow
1. Ingest model specifications for proposed target and draft open-weight models using pandas to structure parameter data.
2. Simulate theoretical latency reductions per decoding step using the formula: Speedup = 1 + (draft_tokens * acceptance_rate) - verification_overhead.
3. Invoke hosted Claude via Anthropic API to analyze memory overhead versus throughput acceleration, validating schemas with Pydantic.
4. Generate a deployment configuration payload specifying speculative draft steps and worker memory allocation.

## Interface
Interactive configuration builder yielding verified YAML parameters for open serving frameworks.

## Data
Benchmark records of token acceptance rates and latency profiles across draft-target open-weight pairings.
Starter examples below are synthetic. They are not downloaded records.

## Source
[vLLM open-model serving](https://raw.githubusercontent.com/vllm-project/vllm/main/README.md)
Leverages vLLM speculative decoding mechanisms (n-gram, draft model serving) to evaluate serving performance.

## Tools
mastra-ai, anthropic, pandas, pydantic.
Mastra AI orchestrates the evaluation workflow, chaining draft model sizing with Claude reasoning tasks.
Hosted Claude accessed through official SDK; draft/target models are open-weights only.

## Deliverables
Submit runnable source, pinned dependencies, setup commands, a working interface, example outputs, and a recorded end-to-end demo. Include tests and an action log.

## Acceptance
- Calculates net speedup factor strictly as: 1 + (draft_tokens * assumed_acceptance_rate) - verification_overhead.
- Accounts for additional memory footprint by summing target size, draft size, and a static 20GB KV-cache overhead.
- Emits valid serving configuration blocks with Claude-generated optimization notes validated by Pydantic.

## Starter Examples
```json
[
  {
    "input": {
      "target_gb": 140,
      "draft_gb": 2,
      "assumed_acceptance_rate": 0.8,
      "draft_tokens": 5,
      "verification_overhead": 0.5
    },
    "expected_output": {
      "speedup_factor": 4.5,
      "required_vram_gb": 162,
      "viable": true,
      "configuration_recommendation": "Use 5 draft tokens per speculative step. Synthetic simplified record."
    }
  },
  {
    "input": {
      "target_gb": 140,
      "draft_gb": 16,
      "assumed_acceptance_rate": 0.6,
      "draft_tokens": 4,
      "verification_overhead": 0.4
    },
    "expected_output": {
      "speedup_factor": 3,
      "required_vram_gb": 176,
      "viable": false,
      "configuration_recommendation": "Memory requirement exceeds limits. Synthetic simplified record."
    }
  }
]
```

## Safety
Simulates speculative decoding configurations without deploying unmonitored local serving daemons.
Use mock external writes. Require human review before any real deployment.