Spectator mode keeps freezing for a few seconds every 30 seconds or so

I have had this problem for months, but for some reason when ever I spectate a friends game the spectator mode will keep freezing. It is really annoying and ends up putting the game even farther behind than the already 3 minute delay. I tried a full repair and that didn't work. Does anyone know what could be causing this?