First of all, it's not infinite. At max rank and 40% CDR his E cooldown is 5 seconds, and that is late game which takes a noticeable of time since usually Jax maxes his E last.
At rank 1 with 40% CDR it's 10 seconds cooldown.
Only issue is that item like Spear of Shojin exists, but then again it's a bad item for it's value. It has high combine cost, poor build path and it doesn't offer enough stats for it's value, so you are really only buying for it's Awakened Dragon passive.
Looking at the match you had, your team was really poor against Jax.
Twisted Fate: Relies on the stun card, which Jax can dodge.
Miss Fortune: Immobile and ult is interruptable by a stun, which Jax has.
Sona: Squishy and if misses her ult she is pretty much dead.
Riven and Nunu were fine, but your team really lacked proper hard CC that could screw Jax over.
Moreover, the Jax's build was really awful, but then again it's low elo so what can I say.
Jax went blue smite and Bloodrazor, which makes him really squishy. Then he built Triforce which is the correct choice. Then he went spear, which is fine since he was ahead but otherwise again, a poor cost effectiveness. Then he went Bloodthirster which isn't a good item for Jax, and lastly a Phantom Dancer which as well isn't that good, especially when an item like Sterak's Gage exists which is a far more better item for Jax.
Your team was going to get outscaled anyways, so you needed to win that game early. To conclude, Jax isn't the issue, Spear of Shojin is. Other champions like Riven and Renekton abuse it as well. If Jax was going to get nerfed again, he would become even worse of a pick in high elo, as he is already highly counter-pickable.
