Back to evaluations
Public evaluation
catalog_sync_eval
Measures catalog conflict resolution latency and accuracy under high update throughput.
Evaluation type
task based
Challenge
Build Real-Time Catalog Sync Agents with AI SDK and Blaxel
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
catalog_sync_eval
Sends parallel catalog updates to the agent runtime and checks output reconciliation state.
Input format
JSON object with conflicting stock records
Output format
JSON object with resolved master stock record