There should be an option to uninvite a friend to a queue.

Sometimes I just accidentally will click a random person who added me, and I feel bad having to kick them. It would be a lot easier if there were just a way to uninvite someone.