Would the 12% decrease in damage from Ninja tabi take away from Thornmail damage?

No, because thornmail damage is neither an auto attack nor physical damage.

I'm convinced thornmail is a placebo effect item.

The only person it tricked into working was Rammus. But Rammus don't care, he never really cared. He was just okay getting something nice.

Yes, it takes from thornmail damage.

Thornmail's return damage is calculated before damage mitigation applies.

It wouldn't be affected.