Stuck in queue- can't matchmake :(

When I'm trying to get into a game with friends and we are in the matchmaking queue, we end up sitting in queue for forever. But when I drop from the queue, my friends will suddenly get matchmade, and when they accept, it says that I had declined. Because of that happening a few times in a row, I am now placed in a lower priority queue. PLEASE HELP!