Architecting Autonomous Refactoring Pipelines with CrewAI and Claude Opus 4.6.6
Leverage the power of Claude Opus 4.6.6 and CrewAI to simulate the architectural refactoring of complex codebases. Inspired by the rapid migration of legacy systems to memory-safe environments, this challenge requires you to build a team of agents that analyze structural dependencies, propose modernization strategies, and validate refactored code modules against strict performance requirements. You will utilize Pydantic AI for structured data validation across the agent team and incorporate Lakera to ensure that code-generation prompts are hardened against injection attacks. Evaluation is integrated directly into the development cycle using Galileo to monitor agent performance and drift, ensuring that the autonomous migration remains consistent with your initial codebase standards. By integrating All Hands AI as a development interface, you will provide a seamless environment for the agents to interact with live repositories, ensuring the refactoring process is iterative and measurable.
What you are building
The core problem, expected build, and operating context for this challenge.
Leverage the power of Claude Opus 4.6.6 and CrewAI to simulate the architectural refactoring of complex codebases. Inspired by the rapid migration of legacy systems to memory-safe environments, this challenge requires you to build a team of agents that analyze structural dependencies, propose modernization strategies, and validate refactored code modules against strict performance requirements. You will utilize Pydantic AI for structured data validation across the agent team and incorporate Lakera to ensure that code-generation prompts are hardened against injection attacks. Evaluation is integrated directly into the development cycle using Galileo to monitor agent performance and drift, ensuring that the autonomous migration remains consistent with your initial codebase standards. By integrating All Hands AI as a development interface, you will provide a seamless environment for the agents to interact with live repositories, ensuring the refactoring process is iterative and measurable.
Shared data for this challenge
Review public datasets and any private uploads tied to your build.
How submissions are scored
These dimensions define what the evaluator checks and which criteria separate a passable run from a strong one.
injection_scan
Ensure Lakera flags zero vulnerabilities
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
refactor_accuracy
Success rate of syntax-valid migrations • target: 0.95 • range: 0-1
This dimension contributes its full weight only when the submission satisfies the requirement. Partial credit is not awarded.
What you should walk away with
Master CrewAI role-based orchestration to decompose monolith refactoring tasks into granular assignments
Implement Pydantic AI models to guarantee data integrity in agent-to-agent communication
Deploy Lakera scanners to audit prompt-response cycles for security vulnerabilities during the refactoring process
Integrate Galileo observability to track latency and quality metrics for autonomous agent cycles
Configure All Hands AI as the interactive agent execution environment for multi-turn coding sessions
Design task-specific goal metrics to optimize Claude Opus 4.6.6 reasoning outputs for complex migration logic
How this agent runs
Validation of code structure integrity and security policy compliance.
Challenge input
Source codebase snapshot
CrewAI
Framework for orchestrating
Galileo
Generative AI eval and observability platform.
Evaluated output
Refactored module and architectural plan
- Ensure Lakera flags zero vulnerabilities
- Success rate of syntax-valid migrations • target: 0.95 • range: 0-1
- Refactor Accuracy target: 0.95
- 1 public reference case
- Python execution harness
View technical recipe
Configured tools
- CrewAI · Required
- crewAI · Optional
- Galileo · Optional
- Galileo · Optional
Evaluation contract
- injection_scan · Weight 1
- refactor_accuracy · Weight 1
Recipe state
This is a preview. The configuration can change before the evaluation recipe is locked.
Run this agent on your dataset and AI stack
Bring your dataset, model providers, and success criteria. We will scope the right managed run for your team.
Scope a managed run[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json
Requires VERSALIST_API_KEY. Works with any MCP-aware editor.
DocsFind another challenge
Jump to a random challenge when you want a fresh benchmark or a different problem space.