Primarily because physical damage is sustained (and often complemented with lifesteal), while magical damage tends to be bursty.
If you're a marksman and attack someone with Thornmail a couple times, you can be like "ow, I'm taking damage, maybe I should disengage", whereas with the AP version someone like LB or Veigar would unload their combo on someone and delete themselves unless they had also built a lot of MR and/or SV.
There is similar reasoning as to why SV is not as available as LS; burst damage that also comes with massive self-heal is very hard to balance. As a non-League example, see Warlocks in WoW PvP -- Death Coil (which is a long-cooldown instant nuke with large self-heal which also applies a short hard CC) makes it VERY VERY difficult to duel a warlock with it up unless you're something like a Rogue that can stunlock/burst them down. Even if you catch them off-guard and take off a big chunk of their health, they push one button and seriously hurt you and heal themselves, then start laying on DOTs.