Commit

0316200

Adding the moving average dropped it by only 0.009 bpb. It does have an effect, but I don't think it's big enough to say it replaced attention.
Translated from Japanese

commit 0316200