Anybody got thoughts on https://openai.com/index/introducing-gpt-6-1-terra/?
Commit
a95bc58
I have two.
1. They claim Terra beats Astra? I'm confused. Isn't Astra supposed to be their top model?
2. They acknowledge that they still can't beat Opus. Maybe Opus is more expensive. But max 5x (100/month) takes you a long way. Astra drains my weekly usage like crazy.
I didn't see them saying it beat Astra, but it did come close for much lower cost than Astra or Sol. And Opus is a Sol-class model so that fact that it was close is interesting.
Once again the overall feeling I am left with is that the cost per unit of intelligence is falling…
Right, they don't directly claim that Terra beats Astra, but isn't that what the chart shows? A higher score, for less money? Higher is better, right?
Agreed that the falling cost is a good thing. Makes you wonder about the valuation of Anthropic at IPO though. Earning that back, will only get harder if prices are going down.
commit a95bc58
Unless I didn't look properly, I meant that it's not higher on everything. At higher effort and reasoning levels, for more complex tasks, Astra is better.
Also, these benchmarks are becoming less and less reflective of real-world use.
I'm just glad I'm not in this arms race.…