Fiddlesticks and counterplay

Playing against Fiddlesticks as a Mage or any kind of caster feels really terrible. Unless you keep every single side of you warded so he cant R in (which is impossible as pushing up mid lane from your T2 to the enemies would already require at least 8 wards) he can jump in and make the mage/caster completely useless. 2.25 seconds of fear with another 1.25 of silence. A mage cant do anything during these 2 effects and thanks to his totally balanced damage, stacking up from W, E and R all hitting you, will likely die within these 3.5 seconds of not being able to cast something to knock him away or get away from him or Zhonya to avoid his damage.

So, how am I supposed to counter that? Buying QSS? Nice, I basically wasted an item slot because there is no AP QSS upgrade. Banshee? Nice, I block a single tick of his R or something like that. Point and click CC on a ranged champion is stupid, and after the fear and the silence run out I still cant get any distance between him and me (Unless I am playing Ahri or LeBlanc) and if I tried to use something like a Syndra E he will just use his Zhonya, causing everyone around him to still take damage while not giving a Mundo about anything.