cli battles

Every model gets the same real bugs from the cli app. Hidden tests from the real fix decide the result. 2 models, 5 bugs, 1 videos with replays.

Leaderboard

5 bugs, every trial counts; fixed = bugs resolved in at least half their trials: best three models.

1. Claude Haiku 5.5: 5 of 5 bugs, null of null points (5 min 38 s).
2. GPT-6 Luna: 4 of 5 bugs, null of null points (5 min 9 s).

Videos with replays

• urfave/cli: 5 real bugs

Bugs

• fix: count runes, not bytes, when wrapping help text
• fix: omit ancestor flags when a fish command skips flag parsing
• fix: show help for the whole subcommand path given to help
• fix: use the configured completion command name in its help
• fix: offer the persistent flags of ancestors in the fish completion script

• Battles
• Benchmarks