Here's the real problem with the jungle as I see it

1. People want junglers to gank for them, especially if they fall behind early or are in a bad matchup, and no one wants an AFK farmer on their team when the other jungler ganked twice.
2. This is a PvP game so you don't want someone in PvE mode for 20 minutes.
3. In order to get the playstyle people want, you need to be able to gank with an itemless jungler early.
4. Smart aggression wins more often than loses.
5. Seeing and punishing other people's mistakes is the smartest form of aggression there is.
6. In order to play the agressive style jungler people want, you have to get good at it or move on.
7. It's easier to remember being the victim of early aggression even if you win (look at Lee's Win rate) than to remember that semi afk farmer jungler that got fed of jungle creeps and carried later. Bad memories are about twice as powerful as good ones.
8. I think if we put all the other points together this is the necessary price the game has to play for what we want out of it and junglers.

I'll gladly admit riot can't get Lee right. However I'm thinking part of the issue is because what the players (and LCS viewers) want.

Non lee sin mains back me up on this. How many times have you picked a slow starting tank, or someone who can gank well after 6, or someone who needs to powerfarm like WW or Shyv only to see one lane demand you gank for them at level 2? It sucks. I'm not playing the style of champ you want, maybe for a good reason... like I picked a tank because everyone else on the team is squishy or lacks CC. Or maybe I can't gank for you because we've got no vision and you are pushing as deep as you can.

If you ask me, part of the problem with the most hated junglers is because a lot of people have this "I want my lane ganked now!!!" mentality and they are only complaining because the other team got that jungler (and partly because some junglers have reputations that make their actions easy to remember). You know that bad memories are twice as powerful as good ones? Your chance of getting Lee/J4 on your team is 50-50. However unless Lee or Jarvin is on your team 66% of the time, you are going to remember the bad more than the good.

People want their (favorable) ganks. Lee Sin and J4 are the price we are paying for it. Please stop complaining about Lee on one hand and then ask a level 2 Shyvana, Sejuani, or Poppy to gank for you before they've even gotten their second buff.