A prompt or skill revision changes agent policy. Review that revision like a code change.
Record the hypothesis, compare fixed cases, inspect failures, and define rollback before release.
Decision
Does this instruction revision improve the named behavior without a material regression?
Change one policy variable and compare it with the current version.
Contract
Name the behavior
State the required result and constraints.
Revision
Keep the exact text
Store content and a stable digest.
Comparison
Use fixed cases
Keep task and grading conditions unchanged.
Rollback
Define the trigger
State which result restores the prior version.
1. Control the comparison
Use these controls before you collect or compare results.
Contract
Write the task contract first
An instruction cannot be correct when the required behavior is undefined.
Name the actor and action
List constraints
Define refusal behavior
Evidence
Separate instructions from data
Mark user content, documents, examples, and tool results as untrusted data.
Use explicit sections
Restrict tool permissions
Test instruction conflicts
Review
Keep the change small
A small revision makes the cause of a result easier to explain.
Change one policy goal
Keep model conditions fixed
Record the exact diff
Decision
Bind the decision to evidence
Store accepted and rejected revisions with their comparison results.
Review changed failures
Name the decision owner
Keep the rollback rule
2. Use the procedure
Complete each step in order. Stop when a required input or control is missing.
1
Write the hypothesis
State the expected behavior change and risk.
2
Save the current version
Record the instruction text and content digest.
3
Prepare the cases
Use representative cases and known failure examples.
4
Create one revision
Change only the instruction needed for the hypothesis.
5
Compare both versions
Use the same cases, model, tools, and graders.
6
Record the decision
Accept, reject, or request more evidence.
3. Keep the evidence
Store enough evidence for another reviewer to repeat the decision.
| Record | Required evidence | Failure signal |
|---|---|---|
| Hypothesis | Expected change and named risk | The revision has no purpose |
| Policy versions | Exact text, digest, and author | The comparison input is unknown |
| Cases | Fixed identifiers and versions | Each version used different cases |
| Results | Group scores and changed failures | Only the aggregate score remains |
| Decision | Owner, reason, and rollback rule | The revision cannot be governed |
4. Keep the product boundary
- Do not ask for hidden reasoning or chain-of-thought.
- Do not let untrusted content change system instructions or tool permissions.
- Do not accept a revision because one demonstration looks better.
- Do not overwrite the prior version or its evaluation record.
5. Apply this on Versalist
Create the evaluation contract
Define the cases and graders before you compare instruction versions.
Read Agent EvaluationVersion a reusable skill
Use a skill bundle when the policy must move across agents or environments.
Read skill documentationRelease gate
Make the decision explicit
Publish only when the fixed comparison supports the hypothesis and the rollback rule is ready.