On LibraryUseBench, every model hand-writes something the library already does, in 73% to 100% of its solutions. Even the best ones do it in nearly 3 of 4 solutions written. It will become ever more important to solve this as agents increasingly work with agent written code over human written code.
https://gabeorlanski.github.io/posts/slop-moves-up/
gabeorlanski.github.ioSlop Is Moving Up the Stack | Gabriel OrlanskiAgents write cleaner lines of code than a year ago. The slop moved into design: hand-written copies of libraries, catch-all error handling, and libraries other agents don't use.
commit 19ce792