an interesting write-up on Terminal-Bench 4.0
https://preferencemodel.com/blog/terminal-bench-4-is-saturated/
Preference ModelTerminal Bench 4 Is SaturatedWe audited Terminal Bench 4 and found models cheating their way to full marks and tasks with design issues. Here is what we found and how we fixed it.
commit ddc0eb1