So this is an awesome topic, and one that's near and dear to my heart.
There are a few main tricks, in my experience. Your personal mileage may vary, because this is, often, down to your ability to judge the situation and adapt your communication style to the players in your game.
Be Calm and Mute Only As A Last Resort, But Learn When to Disengage
This one is tricky, as you have to be able to deal with seeing aggression and insults without resorting to attacking back, but it keeps open the chance to communicate -- something I don't do unless I see an opportunity, as doing it at the wrong time can frustrate people more. The only time I mute a player is when it's made clear that they are not interested in working with the rest of the team.
Encourage Others to Mute As Needed
If someone else is bothered, saying "just mute -- I'll send you anything important" is a good way to diffuse the situation without appearing to be a villain to either party. It also immediately lets the players involved know that you're prepared to step up to fill a hole in the team communication.
Be Encouraging
Try to temper your comments -- even criticism -- with encouragement. If someone dies and looks like they're going to tilt, a simple "Hey, we've got this. We outscale them hard. I'll try to be top if I see a good chance, and just hang back a bit until then" can often help a player remain in control and feel like you have their back, whether through gameplay or just metaphorically.
Make Calls, But Also Take Calls and Take Risks
No one likes to feel like they have no control. Making calls is great, but if someone else is also interested in making calls you should work with them. Is a toxic player pinging mid lane over and over when you think you should threaten bot? Take a moment to consider if bot really IS the right call, or if telling everyone "Pivot mid, guys. Let's support Yasuo and get that tower down" might not be the better call. Remember -- a call can be the best call gameplay-wise, but a call that helps the team morale may sometimes lead to better long-term gains. I've seen teams rally around a clutch Dragon steal that shouldn't have worked much more often than I've seen teams rally around playing it safe in their jungle, which just feels bad for everyone.
Accept That You Will Not Always Succeed
Some players just don't want to work as a team, or just want to rage. In these cases, mute them and focus on keeping your team as upbeat and cheerful as possible. I like to set new goals in this situation: "Let's take mid tower and kill that Legendary Twitch" is a good example. It means you can work towards things other than watching the Nexus explode. In fact, I like to do this in general -- focus your team on the next objective rather than on the victory, and you get more milestones to make people feel positive about, and more places to unite as a team.
