The attack speed on ghost blade.
Frankly, I think this is the problem. 90% of champions this item was intended for don't care about the attack speed, they care a lot about the move speed, CDR, and pen.
THe reasons adc's can build this item is because not only do they get damage they get attack speed too.
Why build a crit item, when you can build ghost blade and get almost the same attack speed and more consistency in damage, rather than waiting for 100 stacks on shiv/ a lucky crit.
If it was up to me, I would completely remove the attack speed from the item and add some other usefull stat in the active, like more move speed, or some onhit damage that scales with the enemies missing hp.(like 15 onhit damage increased by 1% for every 1% of hp the enemy is missing,max 29 damage onhit) (yes it will make some unpopular champs like twitch stop buying it, but thats the whole point) But this will open itemization choices for non crit champions who want damage, like warwick .
Just my opinion.