Here's the difference:
A tower can be played around and tower diving should be high risk high reward. Right now it's pretty low risk for the reward and it's almost guaranteed.
Stopwatch is a Zhonya's effect, an effect that generally has its counterplay in teamfighting where your team can position up around the person or force the enemy team to engage on something they're not ready for.
Stopwatch in lane is literally always 1v1 and so is an immediate chance to remove any and all interaction with the user whatsoever. It is an immediate "fuck you" to any and all risk taking. If the enemy has it, it immediately means going in is almost guaranteed to go wrong while that enemy can instead use it to save themselves. This makes the dominant strategy always having stopwatch.
Towers are also readily available to everyone while Stopwatch isn't - you have to invest either runes or gold in it. That's why stopwatch should be deleted.