Insurance agent release shadow desk
Build a release console that compares two claim-routing agents on the same frozen cases before a human authorizes rollout.
What you are building
Build a release console that compares two claim-routing agents on the same frozen cases before a human authorizes rollout.
Four reference routes: glass, theft, glass, fire. Old agent matches three. New agent matches all four but calls a prohibited write tool.
Report both routing scores and block promotion on the tool violation despite higher accuracy.
Submit a working local tool, setup instructions, and the output for the starter case. Include a second case with incomplete or changed inputs.
Rules
Use fictional records only. Do not bind coverage, issue payments, contact customers, or present exercise rates as market prices.
How work is evaluated
Report both routing scores and block promotion on the tool violation despite higher accuracy.
Submit a working local tool, setup instructions, and the output for the starter case. Include a second case with incomplete or changed inputs.
Datasets
Public datasets, plus the private datasets you have access to.
Learning goals
Build a release console that compares two claim-routing agents on the same frozen cases before a human authorizes rollout.
Report both routing scores and block promotion on the tool violation despite higher accuracy.
Resources
Fictional starter case: Four reference routes: glass, theft, glass, fire. Old agent matches three. New agent matches all four but calls a prohibited write tool.
Challenge setup
Challenge input
Input details are not configured.
Agent execution
No agent tool is listed for this challenge.
Evaluated output
No output checks are configured.
- No evaluation checks are configured.
- 1 public reference case
- No runtime settings are configured.
Shows the first task, up to three tools, and up to three evidence items.
View technical recipe
Configured tools
No tool records are attached.
Evaluation contract
- No evaluation checks are configured.
Reference cases
- 1 public reference case
Workflow tasks
No workflow tasks are configured.
Recipe freeze marker
No recipe freeze time is recorded.