This is actually a topic that's been under a lot of consideration for a very long time. When I was first learning to jungle a friend showed me a video of a couple of Korean pros discussing jungling tactics, and I was introduced to the idea of carnivorous and herbivorous junglers. Essentially, a carnivorous jungler is someone like Kha'Zix, Rengar, or Lee Sin, someone who excells in early damage and can make his herbivorous enemy's life hell by constantly going on the offensive in the enemy jungle. This is in contrast to herbivorous junglers who need a little bit more time to get rolling, so spend more time "grazing" (farming) like Amumu, a Diana, possibly even a cautious Volibear. Ones who don't have a whole lot of 1v1 potential but by stacking up their other abilities they can make up for it through the mid and late game with a wealth of CC or by hitting their stride in terms of damage later on in the game.
So along these lines, the limitations of aggressive jungling come when you're up against a better predator. For instance, I wouldn't take a Rengar into a Kha'Zix's jungle because, yes, while Rengar can be an amazing counter jungler, early on his damage can be kinda meh overall. Good enough to let him punish weaker junglers, but against someone like Kha'Zix, who's entire kit revolves around singling out targets and making them eat their own terrible positioning choices, going into Kha's jungle could be a death sentence. You could hope he doesn't have the bush you're hiding in warded and wait until he's low enough fighting a camp to hop on him, but you're still running a bit of a risk. Counterjungling can be calculated, yes, but trying to act like there aren't so many other variables in play that can just as easily mess you up as help you is just being ignorant. You can't guarantee where the enemy will be and when. You could think they started red and invade their blue jungle to prep up to take them down only to find that Gromp and Blue are both gone and you've just been tagged by the Wolves' sentry.
You also don't want to take too much time traipsing around the enemy's jungle, because remember that even if you're stealing the odd camp or two they're still clearing their own. You're putting a hitch in their giddyup, to be sure, but if you spend too much time waiting around in their jungle instead of returning to your own to farm your camps, you're going to fall behind. There's a line where you go from being aggressive to just being lazy, and you'll get punished for it.
Ideally, an aggressive jungler should be looking out for ward coverage in the high priority ward locations. Early on in the game, where a more carnivorous jungler can utterly shut down an herbivorous one, no one has the money for a mass amount of wards. After their first back they might buy a sight ward or two, but for the most part you're going to be dealing with trinkets early on. If you're planning to go aggressive, take a Sweeping Lens and systematically take down enemy wards. You're making them waste gold, or simply burn up their trinket, when you pop a Lens over the bushes near buffs or tribushes and take down their ward. You also start playing a mind game with them, where they see you pop up long enough to lose vision as you disable their ward. They know you're there and will start playing more cautiously, and that's what you want. Hesitation. You want your enemy to feel that they're being watched. At any moment Rengar could pounce from a bush, Vi could Q over a wall, Jarvan could E Q, you'll hear that fatal "HIKKUH" as Lee Sin plants his foot in your face with an alarming accuracy that even Daredevil wishes he could have.
Items that support early aggression are pretty straightforwards. Does it increase your clear/dueling potential? Awesome, take it. With the new iteration of jungle items, this becomes a bit more of a game of chess where you can focus more on targeting the enemy directly with an item like Skirmisher's Saber, which can now pretty safely be taken with champions who aren't strictly AA focused, or target them indirectly with an item like Poacher's Knife, trying to do the opposite and be where they aren't by taking camps before they can get there and setting them behind in that way. Early aggression is decided by much smaller numbers than you'd think. A duel can be settled by who only has two longswords, or who has the brutalizer finished.
What champs are not good when played aggressively is probably a bit more interesting to delve into. From an objective standpoint, every champion can be played aggressively if you're good about it. Hell I've been counterjungled by an Amumu who waited until I was almost pitifully low before revealing himself via a Bandage Toss from the WEIRDEST angle, one that I didn't even know he could land one, and taking me down while he was still at about half his health and I couldn't carve away at him fast enough with an Elder Lizard bearing down on me. But within the pool of junglers, some will still have more potential than others. The basics I've already mentioned originally are probably your strongest counterjungling options, even if Kha's been pretty severely weakened early on (don't let that think he won't still end up 27/2/12 off of a few early kills). The strongest aggressive junglers are ones who mix survivability in with damage. Someone who has a shield like Lee Sin or Jarvan, while also deal a significant amount of burst damage (Lee's Q E E Q, Jarvan's E Q W AA(passive proc)) are probably the best options. They can mitigate your early damage, which is probably meh by comparison, while laying out a pretty good sized chunk of hurt on their own. After that it becomes a battle of auto attacks and maybe one or two more spells, but either it's going to end with someone flashing over a wall or someone dying. On the other hand a Vi can counterjungle by virtue of significantly weakening the enemy, her Dented Blows stacks, while using mechanics like AA resets to get in more hits than someone else would. The individual damage she's doing isn't particularly bursty, but she's comboing enough together to make it really add up. A Vi Qing into you, AA E (Denting Blows procs here) AA E AA (since she's got more AS here) is probably enough to bring anyone who was at low health from fighting a buff either dangerously low, or completely dead.
Conversely, the champions who aren't good when played aggressively is pretty much any tank or AP jungler. Tanks usually need to invest in resistances and health before they can hit some sort of dueling potential, mitigating enough damage to let their lesser amount of damage stack up over a longer period of time for a win. AP junglers have a high damage potential, but are incredibly squishy and usually need to invest in quite a bit of AP before they can do enough damage to kill someone before they get axed themselves. The tradeoff here is insane mid and lategame presence depending on who you're playing (My Rammus can pretty routinely 1v3 and leave with a triple late game) whether through sheer unkillable... ness... ocity, or through the amount of damage they can put out.
I feel it necessary to bring up that there's a group in and amongst this that kind of fall somewhere to one side or the other depending very heavily on how they're played. Auto attack/on hit reliant champions (Udyr, Yi, Shyvana, Nocturne, Skarner, Volibear) are kind of an interesting breed. Most can be played either aggressively or passively, but you can't say for certain. For instance, Nocturne is my main, judge me how you will. I have several mains for the jungle, but he's just the one that I've learned to the point where a Red actually complimented me breaking him down and reviewing tips and tricks for someone who posted on the boards about him. The standard strategy when you see an enemy Nocturne is to try and bully his laners before he hits level 6, where his gank/countergank potential skyrockets. And in most cases this holds true. Nocturne will usually wait for 6 to start his aggressiveness, but he can still make plays long before that. I've snowballed entire games by coming down to gank the bottom lane at level 4, when I have all of my abilities available, and landed a kill for myself and a kill for my ADC that we both took off running with. Nocturne can also be a deceptively potent counterjungler as well; traveling from your jungle to the enemy's pretty much guarantees your passive will be available by the time you're in position, an early Q does a solid hunk of damage while giving you essentially a free longsword's worth of damage, and when champions are running on very few available ability casts the chance to soak up one of their best spells into your spellshield while amping up your damage by doubling the attack speed bonus from W while you fear them so they can't even retaliate during the moments you're tearing them apart makes him horrific. But not many people see that. A Volibear who buys a Devourer's instead of Juggernaut can actually outdo quite a few people, and my tank Volibear can usually 1v2 depending on how well I'm doing because I'll sacrifice that early health for a small amount of extra damage with stupid amounts of synergy with his kit, letting him outlast people just long enough to kill them first. Champions like these are all entirely in how you play them, as aggressive as you make them depending on how well you know them.
Now personally I don't much like aggressive jungling. To me it feels a bit risky, and I'm alright with a safer early game because I feel it sets me up for more consistent mid and late games, although I've benefitted plenty from early aggression. If you're dead set on it, the simplest tip is vision control. Get rid of theirs, and put yours up. Learn your champ's capabilities in and out, and learn how to predict what others will do based on their champions. Some may just cut and run entirely, leaving the camp to you to save their own life. Be a bully, and let them know that you're an ever present threat even if you aren't even trying to mess with them at that moment. But keep up with your own farm, and don't leave your laners hanging for too long when ganks could win lanes.
Good luck!
, flat runes, and damage buff masteries, especially on champions like 
, if I plan on scaling into the mid and late game I'll take more "controlled" jungles like 
, and I avoid junglers like 
with little to no early game, or champions needing farm or teammate support.