Baron buff makes skill shot mages useless while ADCs are unaffected
Baron buff makes minions almost unkilkable. The thing is, minions blocking skill shots such as Taliyah Q is huge counter play. So when its late game and you can't get to use your skills as a mage, because Baron buffed minions are blocking you, you become near useless.
Meanwhile, ADC auto attacks aren't affected by anything.