You have declined.

Sometimes for a reason i can't tell after you accept a match the timer will go all the way down then say "You have declined" when you did click accept and take you out of matchmaking.