Punished for nothing?

I was playing a game and I left for like 2 minutes to help my brother with the groceries, then I come back everything's fine theres a warning that if i dont come back ill be kicked all fine right, but when I finished the game I was put into a low priority queue for 20 minutes! This is the 3rd time I've been put into low priority queue and most of the time I have to do something Irl but i usually come back, or my connection quits and I cant reconnect, i almost never rage quit, Is there anyway I can get this removed? And before you go referring me to the leaverbust faq page ive already read it. But is it possible to get banned for this? Cause I really cant help it 95% of the time. :/