That would hurt his waveclear. Twitch uses Venom Cask+Contaminate on minion waves to clear them out quickly.
The reason Twitch is weak is because he has a crippling early game. Notice how, whenever the pros used him, he was almost always lane swapped? That's because Twitch is really bad in the 2v2 matchup. He lacks the range to bully or farm safely, and his stealth can't be used to escape anymore, which leaves him really immobile, not to mention his abilities sap his mana pretty quickly (Not as bad as most ADCs, but enough for it to make a difference).
However, if you can get through that early game and get to his two item breaking point (BortK and Youmuu's), he becomes the best teamfighting ADC in the game. Ever been in a jungle fight against a Twitch? Or in a Jarvan ult against a Twitch? His multi-target damage is insane.
Twitch is also good at killing Dragon and Baron. Against enemies that stay alive so long, his true damage poison really gets a lot of time to tick away. So, the reason Twitch is generally bad is his early game.
Or, you can circumvent that early game and take him into the jungle. My preference.