I think the problem is your understanding of how Armor works is incorrect (don't worry, I made a similar mistake when I got started playing his game). Armor decreases the percentage of damage you take. Take a look at this from the wiki:
Every point of armor requires a unit to take 1% more of its maximum health in physical damage to be killed. This is called effective health.
Example: A unit with 60 armor has 60% more of its maximum health in effective health, so if the unit has 1000 maximum health, it will take 1600 physical damage to kill it.
Armor penetration allows you to ignore some of that armor, and makes your damage more efficient. For example, if the enemy has 100 armor, and you have 10 armor penetration, then your attacks will damage them as if they only had 90 armor. Whether or not this is more gold efficient than stacking pure damage depends on how much armor the enemy has stacked. If they have 30 armor, then they are still taking a large percentage of your damage even without Armor penetration. If they have 300 armor, you better get some penetration if you wish to even leave a dent.