I don't draw the line at a diving suit changing size. Where I draw the line is people jumping to the conclusion that it should change size when nothing in Nautilus's lore implies that the diving suit/himself changed in size.
I don't know about you, but I think that I'd notice if when I went back to my house I towered over it, yet there's no mention of that. Sailors were terrified when they saw him, yes, but do you think they would stop and give directions to a 20 foot monstrosity making its way through Bilgewater? Lets be honest, if Nautilus from the Cinematic climbed out of the sea and into Bilgewater it would be bombarded with cannon fire, but that didn't happen.
The only way Nautilus's lore fits with all of these descriptions and events is if Nautilus is roughly the size of a man.