The true damage passive applies only when you critically strike, meaning that you would get it only 60% of the time, and even then to only 20% of your damage. Also, targets generally will not have “a lot” of armor after one item. Sunfire Cape gives only 60 armor, which would bring a tank to 100 armor once we include their base stats. A 200 damage crit would deal 100 damage without Infinity Edge or 110 with it. That’s not much more damage, and you wouldn’t even get it very often. Also, against targets that haven’t built armor, the passive is worth even less. Infinity Edge is good later in the game, once you have enough Critical Strike Chance and enemies have enough armor.
I have thought about it. For 1,300 gold, I could buy an actual Zeal instead of finishing Infinity Edge. I would lose 10% Critical Strike Chance and the true damage passive, but I would gain at least 25% Attack Speed, 5% movement speed, and be almost halfway to an item granting me a passive I could really use, like longer range, damage reduction, multi-target, or chain lightning. Finishing the item would still be better than finishing Infinity Edge.