What's the point of kicking inactive players?

Let's say a player has to go afk for a few minutes (they need to poop, answer an important phone call, watch the newest Game of Thrones episode, whatever). Obviously, you shouldn't do that, but sometimes you just have to. And you shouldn't do that, because it hurts your team. So when you come back and are ready to play, and not hurt your team anymore... you find out the game kicked you out, so you have to hurt your team for additional five minutes, till the match loads again. Why? What is the logic behind it?