Kicks me out of games

During champion select, after everyone has chose a champion and the timer is counting down, once the timer reaches zero I get thrown out of the game. I didn't think anything of it until it happened a third time and I got a message that said I have declined too many games. I don't want a penalty or be banned from League when it isn't me who is declining these games. Anybody else or just me?