Match Accept

I click the 'Accept' when the match is found but occasionally it just makes me start at the 'Waiting for other players' for about 2 minutes and I can't do anything. After those excruciating 2 minutes are over, it says that I declined the match. Makes no sense.