estimating MMR and the cold start problem
TLDR: Riot had a good idea implementing a hierarchical model for positional ranks but implemented it for the wrong aspect of their game. They should have instead implemented it for MMR across game types.
Imagine you are put in charge of designing a system to estimate someone's skill at League. You have a wealth of data on many players including performance in solo/duo ranked, flex ranked, and normals. You could choose to (a) consider each queue individually and estimate the individuals skill based solely on the data from that queue, or (b) group all the data together and collectively infer an individual's skill.
Currently, it appears that (a) is implemented in league. It's the only way I can explain the poor matchmaking that I've seen in flex queue. Needless to say, matching up against Diamond players when you and your friends are new-gold is not fun. Positional matchmaking would alleviate some of the problems, but not fix what I think is the fundamental problem: cold starts.
When people start playing flex queue for the first time, the system is in a position to estimate their skill level. This happens in recommendation systems -- when a new user makes a Netflix account, Netflix has to identify what movies or tv shows that person is likely to watch. They can use demographic information to help this process, but it is fundamentally difficult to infer their interests. This same problem makes it very difficult to assign a skill level to smurfs as there is not much data to go on (although smurf detection algorithms can help here).
What you would absolutely never do in a recommendation system is ignore a large amount of your data. This is what (a) does. Riot has prior knowledge of an individual's skill level and it appears they ignore it. I'm aware of potential issues with option (b). You don't want to award someone a rank in flex that they may not have earned. But, what is far worse, is having new players (to the game or to a queue) having their games ruined by a mismatch of team skill level. I would conjecture that this is a primary reason why people quit the game as they are starting out. They queue with friends and get smashed unless their friends play carries.
There is middle ground here. You could try a hierarchical model (c) which gives everyone a base MMR distribution and then you sample MMR values for each individual queue/role. I'm pretty sure this type of model is what they tried with positional ranks (although that system had obvious flaws). Those flaws don't carry over to a hierarchical model for ELO. Estimates of individual queue MMR ratings would shrink towards the overall MMR estimate for a player. This would essentially fix the cold start problem for different queues.
Apologies if any of my assumptions are off-base.
