Is having a bad KP really a bad thing

It depends on what you're doing for your team while not helping them kill the enemy (which in turn makes it safer to take objectives).

I've had times when I've tunnel visioned on towers instead of joining team fights and it was the right call. I've had times when I've done the exact same play and it cost us the game.

Depends on what you're doing.