Why does teambuilder do this?

I noticed something about teambuilder.... When I build a team and not invite anyone, I get teammates really fast. However, when I invite someone, at least one person, getting teammates takes FOREVER. I'm talking 15 minutes for a support that's probably going to be like "I don't like this" and leave. I'm aware of the matchmaking adjustment thingy in the corner. This is not affected by whether or not that is active. I simply only have this problem when I invite someone.... So, what gives??