Accepted Invites Not Adding You to Lobby

There have been occasions where I accept an invite from someone, but I'm not moved into the lobby, but I am getting penalized with leaver penalty for not accepting when the queue breaks. I'd like to record this here in case something happens to my account because of client bugs like this.