Which is more effective?

Depends on the behavior.

Is the behavior minor, and not likely by itself worth a punishment? The report button is better because it adds this to other reports for that behavior and will help build up a case for the system to punish them for. Player Support isn't going to intercede if the player's behavior hasn't been severe/frequent enough.

Is the behavior extreme and likely easily detected by the IFS? The report button is still better for the same reasons above.

Is the behavior both extreme and designed to avoid detection? Then a ticket will likely be better, though you should always use the report button first; it helps build a case against them, improves the system, and people often aren't as clever as they think.

since i can only speak about the euw support: reporting via sup. ticket does absolutely nothing.

Report feature would be more effective because it will alert game masters / moderators on the case. Then they'll have to see how it all went down and what started it. As long the words aren't too vulgar to the point they're attacking the player aggressively than just poke and jester. It shouldn't affect their account. However when racial words, racism and you know the top next level of toxicity. Yeah a time out ban is needed.