I could explain to you why your build doesn't really matter against a Yi, but that isn't the point here.
Long story short, no. The damage is not over the top, because its all that he has. I like to think that all the champions in League fit into a triangle; where the three points on the triangle are damage, utility, and tankiness (roughly) all of the champions fit somewhere in this triangle, with the more versatile ones in the center and the more specialized champions on the ends. So a Lulu would be towards the center, but a yi is all the way on the end.
Yi has nothing but his damage in his entire kit. There is no utility in his entire kit, and his only means of survival (outside of lifesteal, but I don't consider this a good argument because anyone can build lifesteal aka its not a part of his kit) is a heal that can be interrupted/ he cc's himself for the duration of the spell.
Of course, that doesn't mean I don't think there is room for improvement. Its hard enough not being able to slow his movement speed, but in addition he ignores attack speed slows (of which there are very few in the game). Coupled with his ability to dodge anything with alpha strike, and the insane amount of cdr alpha strike can get, Yi can be pretty frustrating to deal with even if you do have the tools to handle him.
Its really his ultimate; that ability has a 10 second duration before he gets a single kill or assist. That is an insanely long amount of time, and in my opinion it needs to be nerfed. He should need to earn his bunny hopping, skill resetting, move and attack speed immune, attack and move speed buff god mode skill; not be given it for an insanely long period of time right from the start. Nerf that, and you actually create a window where a team can successfully out play a fed yi instead of just getting murdered.