Allow us to spectate moments after we joined

I spectate higher rank games a lot just because I like to learn, whether it be builds or jungle camp patterns its always so annoying when I can only go back to when I just joined the spectate. Why is the limit even on there? Doesn't seem like there's a reason unless it has to do with system memory or something like that.

EDIT: Allow us to spectate moments BEFORE* we joined