The punishment for first-timing a Champion in Ranked is the disadvantage and potentially ensuing defeat and LP loss that follows from first-timing that Champion. The only reason one could ever potentially be punished for first-timing a Champion is if it could be proven that they were doing it maliciously in order to lose the game; but that is very, very difficult to prove.
On top of that, restricting anything in Ranked play based off of Champion Mastery is a bad idea. Mastery does not directly equate to skill; I could have MR7 on a Champion and really suck at them since I only got MR7 during a time when they were particularly powerful, or when I got some lucky breaks in matches (example; I have MR7 as Azir, but I'm far from great as him, especially after much of his power was curbed in I think it was 2017.)
Plus, restricting your Ranked roster based on Champion Mastery would also require the Ranked play restrictions to be updated; you'd have to have 12 Champions at roughly MR3 (following your suggestion of 5,000 Mastery Points) to play Ranked Twisted Treeline, and 20 Champions at MR3~ to play Ranked Summoner's Rift. Granted, that's still a small feat compared to other Mastery Requirements proposed, but it's still worth pointing out.
And, of course, you have the issue of Champions of similar playstyle (and mechanical understanding in general). Some people can just pick up and play any Champion without any issue. Others are fast learners. And some may be first-timing a Champion with a similar gameplay pattern to Champions they know (I.E, a Diana main playing Leona Support). Preventing those players from playing what they want to play solely because they don't have high enough Mastery with the Champion is absurdly arbitrary, and does probably about as much harm as it does help.