Que Not Accepting When Accept is Pressed

I've had this happen three times today so far where I have pressed Accept to join a game and it says I have declined.

And do not say it is my internet or computer, both were transmitting data fine to the server at the time, I was talking to people in the client.