Sometimes it does take a long time, but those champions either have new/difficult mechanics that need to be learned or specific playstyles that people need to learn in order to use these champions effectively.
Ivern is a prime example - he's basically a Support in the Jungle, something that simply did not exist in any form close to this before. Plus the fact that he clears the jungle completely different from any Jungler. So it naturally took people way longer to figure out how to play him, how to build him, how to use him effectively.
Kalista also had a completely new gameplay-mechanic in her Jump-Passive and also the Rend-Mechanic so again it's only natural that it takes alot of time to learn and practice her.
Bard, Taliyah, Aurelion Sol, Jhin ... the list goes on.
But the real question is now: What really "new" difficult mechanic does Xayah bring which would be difficult enough to argue she just takes people so long to figure her out?
Her build is nothing really new, there are different ways to build her but she doesn't require some special new build in her role; BotrK or Crit-Build both do fine on her.
And her kit is quite straightforward too; use Q and W abilitys, pump out feathers, position yourself to hit as many when using your E. Maybe it needs a bit of practice to pull it off consistently but I'd say that gets nullified by the people on the other side learning on how to play around it. And thats really the one thing you need to do against Xayah since so much of her power lies in the feathers and her E. Aside from that shes just a Auotattacking ADC.
So I don't think the comparisons with Kalista or Ivern are justified as Xayah is not bringing really new mechanics or playstyles with her.