Well, he's from a bygone era where that speed boost, invisibility, and blind made him extremely slippery. Back in the day, mobility was the exception, not the norm, entire champs would be incapable of damaging when invisible because targeted moves were common, etc.
These days the hate is partially just the meme of hating him from the old days, but the rest of it is that there's STILL not really much you can do about the fact that any time you aren't under tower or following a minion wave every step you take you risk losing 40-60% of your health from a shroom. Even with a few seconds of true sight from oracles (something that is a costly choice to pick up on anyone but support now that green wards are gone) it's nowhere near enough to deal with them.