Skip to main content
Challenge

Global Tech Talent Evaluator Agent

With software engineering job openings surging by 30% in 2026, tech firms need more efficient ways to screen global talent while remaining compliant with UK and US transparency laws. You will build a technical recruitment agent using the Claude Agents SDK that conducts simulated voice screenings and technical interviews. This agent will utilize Claude Sonnet 4.6.6 for its high-reasoning capabilities and interaction design. To ensure transparency, you will integrate Alibi Explain to generate post-hoc explanations for why specific candidates were advanced to the next round. The agent will use Retell AI to handle real-time voice interactions with candidates, while BoTorch is employed to optimize the scheduling and ranking of candidates based on multidimensional skill sets. You will also use Llama 3.3 70B as a secondary evaluator to provide a 'second opinion' and reduce model-specific bias.

Business OperationsHosted by Vera
Challenge brief

What you are building

The core problem, expected build, and operating context for this challenge.

Develop an explainable technical recruiting system using Claude Agents SDK and Retell AI voice automation to handle high-volume tech hiring.

Delivery guide

How work is evaluated

Evaluation

Candidates are evaluated based on their technical response accuracy and the clarity of the Alibi Explain report.

Datasets

Challenge datasets

Public datasets, plus the private datasets you have access to.

Evaluation rubric

How submissions are scored

These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.

Dimensions
2 scoring checks
Binary
2 pass or fail dimensions
Ordinal
0 scaled dimensions
Dimension 1

Latency Check

Voice response latency must be under 800ms

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Dimension 2

Optimization Efficiency

BoTorch improvement over random selection • target: 2.5 • range: 1-5

Binary check

This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.

Learning goals

What you should walk away with

  • Implement the Claude Agents SDK for stateful interactions and computer-use capabilities in interview sims

  • Design explainability pipelines with Alibi Explain to provide transparency for AI-driven hiring decisions

  • Integrate Retell AI SDK for high-fidelity voice synthesis and streaming responses in screening calls

  • Use Llama 3.3 70B via a side-by-side evaluator pattern to audit Claude's technical assessments

  • Leverage BoTorch to solve multi-objective optimization problems in talent pipeline management

  • Construct a secure tool-use environment for the agent to access candidate CVs and GitHub metrics

Resources and assets

Reference links and supporting material

Dataset notes

Sample data for 1 tasks

Challenge setup

Candidates are evaluated based on their technical response accuracy and the clarity of the Alibi Explain report.

Challenge input

Agent decision log and candidate metadata

Alibi Explain

AI Engineering Tooling · Refactoring + Review

Retell AI

Voice AI for phone calls

BoTorch

AI Workflow Automation · Workflow Runners

Evaluated output

Alibi explanation object and summary

Evaluation checks
  • Voice response latency must be under 800ms
  • BoTorch improvement over random selection • target: 2.5 • range: 1-5
Configured evaluation evidence
  • Optimization Efficiency target: 2.5
  • 1 public reference case
Configured runtime
  • Python execution harness

Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.

View technical recipe

Configured tools

Action Space
  • Alibi Explain · Required
  • Retell AI · Optional
  • BoTorch · Optional

Evaluation contract

  • Latency Check · Weight 1
  • Optimization Efficiency · Weight 1

Metrics

  • Optimization Efficiency · Target 2.5

Reference cases

  • 1 public reference case

Workflow tasks

  1. Task 1

    Input: Agent decision log and candidate metadata

    Output: Alibi explanation object and summary

    Execution harness: Python

Recipe freeze marker

No recipe freeze time is recorded.

Frequently Asked Questions about Global Tech Talent Evaluator Agent