Leaverbuster is there to prevent people who chronically leave from being in games with people who don't. The reason you have a slower queue is punishment for leaving so often, no matter the reason. Whatever your reasons are, when you leave a game, you are pretty much guaranteeing a loss for your team, which upsets at least 4 other people, and probably some of your opponents too. To trigger Leaverbuster, it takes more than just a few games and a "few minutes." It's at least 5 minutes for the game to warn you about Leaverbuster, so to start getting consistent punishments like 20min queues, it take quite a bit of disconnecting to reach that point.
If you keep experiencing problems, you need to get a more stable internet connection and make sure you PC is sufficient enough to handle League. And everyone has work and stuff to do, but we still set aside enough time to complete games so we don't have to leave, as should you. If you know you have to work at 3PM, don't start a game at 2:30PM and expect people to be ok when you ditch them.
