To be a mid a good midlaner you need reliable damage, safe waveclear or mobility. Brand's waveclear is not that good compared to the other mages, has no mobility and needs the enemies to be grouped together to do damage with his passive.
I used to play him years ago and he was dealing way more damage. The rework was a massive nerf unless you are playing vs grouped tanks. Now he is a kill lane support and trash mid. Support Brand has no good income and the games are over in 15 minutes with the new surrender. I can't see how is he destroying you. Probably Brand players you have met were decent.