I wonder how much better/worse the site would run if they had their own hardware like StackExchange. And I wonder how StackExchange would run if it were on AWS.
In my experience running large sites, dedicated hardware is not only cheaper but orders of magnitude faster because you can finely-tune the hardware to very, very specific use cases, and you can have a fully private, highly optimized network with very small cable runs to effectively eliminate network latency issues.
Most of my current job involves solving problems that are caused by cloud limitations.
And if you need throughput, 40gigE / 56gigE is not that hard expensive with your own hardware. You end up with like 4x to 10x servers to handle the same load.
I read an article about StackExchange's servers and it seemed super optimized, although I'm not an expert. I can't imagine AWS would be as optimized for exactly StackExchange's requirements. So I think StackExchange would run slower on AWS.
Reddit, on the other hand, is the worst performing out of all the top sites that I can think of.