Loading Screen Blacks Out and Doesn't Let me Connect

I try to load up a game and sometimes at seemingly random times the loading screen turns black and gives me an error message "Unable to connect to the server. Please check your network connection and attempt to reconnect to your game.". This happens multiple times and I'm afraid I will get punished for it. I've tried restarting the client and it doesn't fix it. I've tried the Hextech Repair Tool and it doesn"t work. I've tried restarting my computer too and it doesn't work. The thing is it isn't my internet either since immediately after the game is done I can load up fine. Does anyone know how to fix this and if I will get penalized.