failed to accept ready check bug

I have a reliable wired connection I can't imagine it being a connection issue on my end, I haven't recieved a penalty yet but 2 times now in a couple weeks or so, I have click accept when being prompted to join the lobby, it waits as if other players have not yet accepted and then gives me a message saying 'e46 fanatic has failed to accept blah blah' as if I never clicked it, even though I was one of the first to accept...