decline game without clicking

Every time a try to search for a game and it pops opp it just goes away and sais that no game was found, after i try this 4 times i get a ban pop up that says i have to wait 4 minutes before searching for a new game, and after the ban its normal and i can join a game. i was wondering if anyone else had this problem , if it was common and if there was a fix to it.