rammus is a tank. meaning that some people you just can't solo because most of your kit is designed just living longer and CC. 500 armor is one thing. and if you were the right kind of champion you might be able to duel vayne. Malphite for example with 500-600 armor can duel vayne because he has a crippling attack speed slow. rammus does not. he can only taunt you. which doesnt do anything if you are the only one there.
Vayne is designed to counter building pure armor so the ADC kills themselves. if you are against a vayne as rammus why are you not making sure you have team follow up. rammus is best when he is with his team not only soaking up damage, but being a CC nuisance and doing AOE dmg with his ult (which is actually pretty insane if he gets the full channel off before dying in a team fight. )
im not sure what kind of place rammus is in right now, but I have played him a fair amount recently and he feels "ok" (joke intended). I think the biggest thing he needs is lower cooldown Q and maybe double or even triple dmg to jungle monsters on his defensive ball curl (W) to help him clear as before he gets cinderhulk his clears are some of the slowest in the game. and that is honestly what holds him back.