It's 5 minutes, it's not that long of a wait in the first place. Having the majority decide the vote wouldn't be fair for people that want to play the game out. Surrendering that early when barely anyone has 1 or 2 completed items should be a decision unanimously made by a team.
Why do I have to waste 5 more minutes of my time because on person on the team is Trolling/Tilted/Dumb etc.
What you see as someone trolling could be someone that's actively trying to win the game or make an effort to comeback. Not everyone wants to immediately forfeit when you're down 6 kills or a tower.
Contrary to popular belief, champions still scale, splitpushing or taking objectives still apply map pressure, and comebacks are very much still a possibility.