Is there a reason we couldn't have smite buffs and plants?

Would it have broken the game or something? In my probably worthless opinion, it could've helped a lot of struggling junglers if they could use Gromp poison or Krug stun while clearing the new camps. It's not like plants and smite buffs directly contradict each other, so what was the balance issue?