Finicky Game Invites

Lately it seems if you you are in a lobby, if you attempt to join another person's lobby, a error pops up saying that an error occurred, and to try again later. Leaving whatever lobby you were in and then accepting the invite seem to work.