A high score doesn't tell you what a system actually learned

It's easy to build a challenge that's really just a trivia test wearing an AI costume — one clean input, one clean expected output, done. Those are fine for a sanity check. They're not fine as the thing you use to decide whether a system is actually good, because real tasks are messier than that, and a challenge that hides the mess is teaching you nothing.

Start from a question that doesn't have one clean answer

The challenges worth running are the ones that make a participant deal with uncertainty, incomplete information, or requirements that are in tension with each other — the same conditions the system will actually face outside the leaderboard.

Look at the trade-offs, not just the top-line number

Accuracy is one axis. Robustness, cost, latency, and fairness are others, and depending on the task, one of them might matter more than accuracy does. A challenge that only reports one number is quietly deciding for you which trade-offs don't count.

Make the evidence something a stranger could check

Where did the test data come from. What is the rubric actually measuring. Which failure cases were the ones that mattered enough to include. If you can't answer those three questions about a challenge, the score attached to it isn't worth much.

Versalist gives you the whole loop, not just the ranking

Challenge environments, skills, test cases, rubrics, Episodes — the point of all of it is to use the result to make the system better. Climbing the leaderboard is a side effect, not the goal.