If you got a 20 minute wait time then it means that you have a history of frequently leaving/afking. The system has built-in leniency for unexpected situations but frequent leaving/afking will use that leniency up. To get it back you have to not leave/afk in games and take care of everything that you need to do before you queue up for a game. From the Leaverbuster FAQ:
HOW LONG WILL I REMAIN IN THE LOW PRIORITY QUEUE?
You will be placed in the low priority queue for 5 games. If you continuously leave, the low priority queue timers increase from 5 minutes per game to 20 minutes per game.
IT'S NOT MY FAULT. MY INTERNET/YOUR SERVERS DISCONNECTED ME. I HAD A REAL-LIFE EMERGENCY. I HAD TO GET OFF THE COMPUTER. THE GAME CRASHED. UNBAN ME!
We understand that players may be forced to leave from time to time so the system has been designed to keep these situations in mind. After leaving a few games, you will be placed into a low priority queue; as long as you can complete 5 matchmade games without leaving, you will be returned to the normal queue. However, if you continue to leave games, you will remain in the low priority queue.