The idea is limited by the constraint of not inconveniencing legitimate voters, people who read a post and vote based on their actual opinion. Increasing the iteration time of the action of downvoting discourages the quick repeated behavior of downvote sprees. Granted not by much, but multiplied over 20 or so posts that some troll has gone around down voting it can start becoming a noticeable negative feedback loop.
Other variations of this idea may be better, like a anti-spam measure similar to major chats like Twitch, where exceeding a certain 'rate of downvoting' can diasble your downvote privileges for a temporary amount of time, but that's a whole lot more complicated and I wanted to start basic. Also, it's not a huge deal now I'll admit, but with the vote reliance of post visibility I can see this being an eventual problem for promising and awesome posts getting killed in the crossfire by getting downvoted at the very beginning.