How does Siege Minion aggro work?

Maybe it's just me but it seems as though siege minions keep swapping what minion they are attacking during laning phase