Spectate Client Keeps Freezing

Whenever I spectate my friend's match, randomly at certain points the client will just freeze, and keep going. I've played in live the past couple of days and have no lag at all, but in spectate it lags every other minute, sometimes sooner. Has anyone else had this happen to them?