Lag After Alt-Tabbing

Hello :) Lately if I alt-tab out of the game to look at something and come back, my FPS will drop down to about 30 or so, my FPS counter doesn't show the drop, but it's very obvious that it's there. Maybe it's a memory leak or something in the game, but it's very annoying and honestly unplayable, because it lowers even farther when in team fights and such. Any reason for this? Any way to fix it in game or something? It's only been happening recently, and I haven't changed anything in my PC recently, and it only happens on League of Legends.

Specs - Windows 10 GTX 980Ti FTW i7 7700k 4.2 GHz 16 GB DDR4 RAM Some 5 TB HDD