yup battles

Every model gets the same real bugs from the yup app. Hidden tests from the real fix decide the result. 2 models, 5 bugs, 1 videos with replays.

Leaderboard

5 bugs, every trial counts; fixed = bugs resolved in at least half their trials: best three models.

1. Claude Haiku 5.5: 3 of 5 bugs, null of null points (6 min 38 s).
2. GPT-6 Luna: 2 of 5 bugs, null of null points (5 min 49 s).

Videos with replays

• yup: 5 real bugs

Bugs

• feat: Implement standard schema interface (#2258)
• feat: resolve ref params if present when describing
• fix: lazy validation errors thrown in builders should resolve async like other validations
• feat(string): Create .datetime() (#2087)
• Allow parent resolve before describe. (#2051)

• Battles
• Benchmarks