Is There A Downside To Queue DOdging?

I've seen the queue timer thing, but is there a punishment past that? People seem to be doing it way too much for there to be any real threat of punishment.