There would be two major problems with a thornmail for AP.
Firstly that much MR in a single item would be devastating for burst mages. Yi and Yasuo can do constant damage, even if it is reduced by the thornmail. Burst mages do a burst and they are done. An item that effectively eliminates their burst makes them irrelevant.
Secondly, thornmail deals damage to the attacker. Yi builds lifesteal to counter the effects. The vast majority of mages do not have sustain to counter that damage because spellvamp is largely ineffective. And they don't have much health. Reducing their damage to a pittance and then making them damage themselves effectively eliminates mages as a viable class.
A better solution to the magic resist problem I feel is to rework the Orb of Winter and bring it to the rift. Transform it to a non-tank item (reduce the excessive 70 MR while buffing it in other areas) and make it a viable defense option for any role or class.