Can't press on the map to move camera

I usually play with a finger on my Y button, I lock and unlock the camera constantly during the game. I've noticed that if I do that a lot of times in short periods of time, camera bugs out and then I can't press on the map anymore (camera won't move, and then I have to move the mouse all the way up to the part of the map I want to see) That's it