Back to evaluations
Public evaluation
quantization_eval_task
Evaluates LangGraph workflow output on recommending optimal quantization for specific tasks.
Evaluation type
task based
Challenge
Hardware Quantization Quality and Speed Evaluator with LangChain and LangGraph
Difficulty
Intermediate
Rigor
Unspecified
Evaluation overview
How the linked challenge is judged: tasks, benchmarks, and criteria count.
Tasks
1
Benchmarks
0
Criteria
0
Task templates
Inputs and expected outputs.
Task 1
quantization_eval_task
Ingests task sensitivity specifications and precision metrics, returning optimal precision level.
Input format
JSON containing task_type, accuracy_tolerance_pct, and precision_metrics (FP16, FP8, INT4)
Output format
JSON detailing recommended_precision, memory_saved_gb, throughput_gain_multiplier, and accuracy_delta_pct