Especially with how they pick (one of) the most likely word as the next one. And the most likely word is exactly the one with least entropy, the least surprising one and giving the least amount of information you can.