What if it was harder to keep stacks from fervor of battle?
By making it it automatically reset on ranged after 2 seconds of not damaging an enemy champ and 4 seconds for Melee. Make it require focusing the enemy and sticking to them to keep the stacks. It is currently 6 seconds for both ranged and melee.
If nerfing the damage is out of the question then nerfing the ease of maintaining it's stacks would be another way to make it less powerful.