Actually, you can only get things negative with reduction, not penetration. 45 Magic Pen against a target with 30 MR will only get the effective MR down to 0. 45 MR Reduction would actually drop the target's MR to -15.
However, in today's game, they've made it very difficult to reduce resistances below 0. The order they take reduction/penetration into account makes it that way. It goes:
- Flat Reduction
- Percent Reduction
- Percent Penetration
- Flat Penetration
The only means of bringing a target to negative resistance is flat reduction, which is calculated first. Generally, to reduce resistances below 0, you need reduction from several sources. Very few champs (if any?) can bring somebody to negative resistance values on their own.
That brings up another critical differentiation between reduction and penetration: reduction will actually reduce the numerical resistance value of the target. If you look at their stats, you'll actually see their resistance number decrease when reduction is involved. This means that reduction is shared for any source of incoming damage of that type, not just damage coming from the person providing the reduction. So only one person needs to carry Abyssal Scepter, for example, and it'll help boost the magic damage output from the entire team.
Penetration ignores resistance, it doesn't remove it. So pen is a personal stat that is not shared with allies. It's only handled during the damage calculation, and the target's raw resistance value remains unchanged.