Declining matches when I didn't?

Whenever I search for a match with my friend it looks for a match then kicks us back to lobby with the msg ''X declined this match'', even though I never even got an accept or decline pop up. I'm waiting for an hour because I'm getting the queue dodge penalty. What the hell riot?