Early Jungle Ganks

It's getting really irritating as a top lane player to see the enemy jungler with red buff and top laner walk in to lane together for a cheesy early gank. This happens in about half the games I play and it immediately makes the dynamic of the lane much different. I can play against the enemy top laner, I'm always at the mercy of the enemy jungler as to whether or not I die while getting the first few minion CS. It's getting annoying. It immediately forces flash and over half my health, allowing the enemy top laner to have a substantial lead for the rest of early laning unless I want to back and waste my Teleport too. The main offenders of this are Ivern and Lee Sin, it's almost unavoidable.