It is about knowing how to play around them. If I know what it does and how to screw them, they they aren't toxic nor annoying.
There are way more toxic abilities than Akali/Yasuo's W. Trynda R for an example. If Yasuo or Akali get cc'd, they dead unless they buy QSS to escape, if Trynda gets hit by cc, he R and kill your team.
Something is toxic if it has no counter play, like Trynda R, the only counter play to that skill is to nuke him 100-0 before he can react. CT's champs have the counter play, players just need to do it. You don't counter a CT champ as you do a normal one, and that's what makes them important to the game, it is the fact that they force you to understand mechanics and aspects. If you can't control the wave, you are screwed by a Zoe since she can do whatever she wants.
Just because some champs can't play around a skill, like most adc's against Akali's W for example doesn't mean it lacks counter play. Counter picks are a thing and some champs can't do stuff against others, that's normal. Toxic means that no champion can do anything, like against Udyr or Pantheon or Jax.
What can you do against Udyr? If he's half decent and has a kill, what can you do? You can't duel him, you can't run away from him unless you're Talon or one of the camo champs so he can't know where you went, as 1 wall is many times not enough to get enough space between you two, and if you do get an upper hand, you can't chase him either, where's the counter play?
Same goes for Jax, he's hard to duel, he can burst, sustained damage, strong early, strong lane, strong jungle impact even from level 2, super strong late game, good and healthy clears in the jungle, good chase, good escape since it has barely any cooldown.
Those 2 are toxic champs, as barely any champ can do something against them, as unlike Akali or Yasuo, you can't really play around them, if they get close to you, youz ded, but if Yasuo or Akali get close, you can outplay them.
Toxic doesn't mean "OMFGBBQCT my champ doesn't have too many tools against these champs" but "who the Shaco can do something against those champs?".
Number tuning can be an issue, but that doesn't mean toxic.