"Attempt to join queue failed" ~I never did anything!

This really got on my nerves and I would like this problem to stop happening to me.

The problem is that I got this similar low priority queue time out of nowhere and it has said "Because you abandoned a recent game during champ selection or declined too many games, you are currently unable to join the queue". Simply, I just did the following:

1. Log on League of Legends
2. Hit "Play" and pick one of the two, Ranked or Normal
3. Queue lapse time increases while NO game was found.
4. Cancel queue and search for a game on Ranked. 5 Queue lapse time increases yet again no games found after 3 minutes.
5. Switch back to Normal and then said I had a low priority queue time. This had happened before when I was on queue with a friend and he saw that a game was found while I could not see the options "Accept" or "Decline" Very nerve racking but I was lucky that time when I knew he wasn't joking and I stopped the queue. Please get this problem solved since it isn't fair towards the players that done nothing at all but just waited normally in a queue.