Hey there! I'm actually a Veigar main-ish person in G2 (omw to Plat), and I've had a lot of success building him tanky, particularly into assassin matchups. RoA, Abyssal, and Rylais are incredibly efficient (in my opinion) because, like you said, the AP-stacking can pretty much take care of itself. When you grab Void Staff and Rab Cap you're still breaking 1000 AP easily, and it's pretty great.
I've never played jungle Veigar, and now I'm gonna have to try it in normals. But like you said, I think going straight tank won't be that effective. He doesn't really bring any CC to the table that would encourage him to be on the frontlines - his E has sufficient range to take full advantage of it from his normal spot in the backlines if you position well - and his damage won't be a threat to tanks and he'll be easy to peel off the carries if you try walking at them like a tank. Also, his cleartime would be legit terrible, and as Jungle Nasus taught me in Season 3, stacking q's in the jungle isn't actually as strong as it sounds and you end up consistently getting way fewer stacks than you would in a lane. Overall I think he's better as a kiting damage-tank hybrid mage who gets to farm up in a lane.
But hey, I'm still gonna try it, and Gold 2 isn't actually that high so I could be completely wrong. You should totally keep experimenting with "weird" builds though, because if you're the first one to discover the next AP Tryndamere you get a free ride to Diamond! Yay!