Hitting enemies with abilities could reset the timer on your FoB stacks.
Feels like Fever of Battle might need a little a bit of love right now. Allowing players to keep their stacks up with their abilities rewards the play pattern that FoB is supposed to support.