[Gameplay] Blue buff not attacking

When I attack the blue buff on my first and maybe second clear, the golem will stand still even though I am attacking it. It almost looks like the golem is cancelling its auto attack. It lasts for a couple of seconds, unless I use CC on it. I am playing whenever this happens, but I don't know if this happens with other champions. Even though I don't mind this since it makes for an easier first clear ;) I am guessing this is not supposed to happen.