No, not remove it.
Replace it.
Crit now grants auto-attacks additive %Pen while Last Whisper becomes a heavy flat penetration item. This would allow Crit users to still burn down heavily armored targets without turning soft targets into a fine red mist. It would also remove bullshit like 1% crit chance cheese. Ashe's passive would require some minor reworking, but the coding already exists to make an auto-attack deal double damage outside of Critting so that's hardly an issue. Try could either keep his crit as a unique mechanic like Jax' dodge or it could get remade into simple physical damage amplification.
This would probably require reworking a lot of Ranger kits and generally removing the massive level of homogenization they suffer, but frankly I think that would be a good thing.
While we're at it, %Pen is now calculated after flat Pen and Voidstaff ignores 50% of the target's magic resist that exceeds 60~75 or so.
There. Several really awful problems all solved at once.