Back to evaluations
Public evaluation

cad_assembly_search

Evaluates mechanical assembly search recall and response latency when querying dynamic STEP metadata graph.

Evaluation type
task based
Challenge
CAD Graph Analysis and RAG Search using LangChain and Azure OpenAI
Difficulty
Intermediate
Rigor
Unspecified
Open linked challenge

Evaluation overview

How the linked challenge is judged: tasks, benchmarks, and criteria count.

Tasks
1
Benchmarks
0
Criteria
0

Task templates

Inputs and expected outputs.

Task 1

cad_assembly_search

Processes natural language query regarding sub-assembly dependencies and returns matching part IDs.

Input format

Natural language query string

Output format

JSON list of dynamic matching CAD part identifiers