Really, because the barrels have the most counterplay out of GP's entire kit, you can see the animation of them coming down and there's a fucking countdown that basically lets you know "Hey, I'm about to blow up shit around my immediate vicinity." Plus, if you're in range to get hit by a barrel (not a combo chain, just a single barrel) then you're probably in range to auto it and kill it if you can time it. They're quite a bit easier to deal with than the rest of his abilities, and that's why they hurt more. And to dodge Barrel chains, you just need to position carefully (especially near bushes you don't have vision of) so that you can react if he sets up a chain, and it's easier to dodge if you have a blink or dash.
Also, think about this. Say you gut GP's barrels (Which are a primary core function of his kit) so that the damage is significantly lowered. This could throw him completely out of viability, so instead the team lowered the power of other parts of his kit to compensate for the large amount of damage the barrels do. Not saying GP is 100% balanced, but he's strong when picked against someone who lets him scale for free, his early game is rather slow before Trinity Force.
Also, he's kind of supposed to be able to do damage, he's a late game Carry and if he tries to mostly use Auto's in a teamfight he should probably get blown up assuming the enemy isn't a sack of potatoes.