I think the problem with it is that it lacks an identity...
It makes a LOT of sense as a first item rush. When you're on 1 or 2 items, that +80AP bonus from the active makes a HUGE difference in your pool of AP. That's basically a 30-45% upgrade to your AP (and hence damage). It's like temporarily being 1700g ahead of your opponent (two blasting wands) - again a huge amount when you're only at 1 to 1.5 items.
However, you need to stack it - which takes so much time in lane phase since you're not spamming spells and don't have teamfights everywhere for it to count everyone else's spells. And don't kid yourself, you're not spamming spells because you rushed
and therefore don't have
and have no mana to spam spells.
So consider the alternative: build it 3rd or 4th item.
Except that makes no sense - if you want raw AP,
does SO much more. Likewise
etc... all offer a similar amount of power that is always available, rather than requiring a lot of buildup.
+80AP (1700g worth) isn't that big of a deal when you have 10k total gold (17%). Opposed to when you have 3400g (50%).
So it should be an early game snowball item, but you can't effectively use it at that point.
It's effectively used late game, but the bonus from it matters so much less at that point.
Which is ultimately why you see some Vlads run it (because they don't need mana) and otherwise it doesn't get built.