The enemy taking Dragons feels really bad, because it's not something you can make up.
I understand the flaws with the old Dragon system. But the problem with the new one is... well, say the enemy has a wild Udyr who counterjungles my Amumu hard and takes two Infernal Drakes. I know, right then, that I'm never going to catch up. Even if our team would outscale, we're never going to get the chance to, because even if by some miracle a third Infernal Drake spawns, and we take it, we're still behind.
I mean... kills, towers, gold, all those things you can catch back up in. But Dragons? Nope, enjoy playing at a deficit for the rest of the game.
If the enemy team gets 3 Infernal Drakes, the game's over. It just feels overbearing for something that's supposed to be a bit of a side objective.