Decrease tower diving efficiency mid game

Hello All, I have an issue that early-midgame a jungler with a weak tanky item is able to stomach like 4 blows from the tower no problem and kill the champ in the lane. Now I know the game shouldn't take too long to win but I don't think buffing tower damage in the mid game (11-20min) would significantly increase the game time. I just think its too easy to tower dive and not be punished at all currently. In fact its almost natural to tower dive in a lot of ganks. Naturally as the game progress past the 30-40 min mark the towers should become more vulnerable but they don't discourage tower diving currently in the mid game. Please share your thoughts thanks!