You'll only receive a notification that your report led to someone being punished if you the LAST person to report them before the punishment was weighed. This is definitely something the system could work on improving. Let's say you instantly reported them after the game and queued up again, while someone else took 5 minutes to write down a report on them and attach screenshots to Imgur (or some other hosting site). If they received the punishment after they were reviewed from the second report, you wouldn't receive the notification - the other person would. Usually the IFS (Instant Feedback System) is pretty good at quickly reviewing someone's chat logs and providing the notification, but sometimes a manual review is necessary. Same thing if they queue up again and are punished after someone from the next game reports them - you'll never know it, and the only way you'll likely be able to have a "tell" is by checking their Match History and seeing that they randomly stopped playing for a period of time, as punishments are relatively confidential unless the player chooses to share.
Also, what EvilDustMan said is true. Muting them and reporting them is about your only line of defense in these situations. If you argue with them and become toxic yourself, you're only at risk of being punished yourself. I like to think of the late comedian George Carlin's quote in these situations:
"Never argue with an idiot. They will only bring you down to their level and beat you with experience."