Started with a second reply last night; was too tired to finish. Now I've merged that reply and this one. Here you go:
####A little more in-depth response
Re stomps: The problem with stomps as the primary tool to evaluate matchmaking is that some percentage of games will go this way, regardless of matchmaking quality. This makes judging based off small sample to be of limited use. I am not claiming anything about the current matchmaking quality, just that this approach for evaluating would have some built in difficulties. That's why I think the macro question is the more relevant one - how would you judge if the system is doing a good job or not?
First off, I wasn't judging only based on the outcome. I was also judging based on their subjective general skill level - they seemed to have a better sense of where and how to rotate, when to back off or stick around, were better-positioned, and had more technical skill (with the exception of ezreal).
The difference here is that a stomp could happen because one player gets an advantage and snowballs, even though the lane was evenly matched, or the player could start out with a -skill- advantage and use that to snowball. The former is fine; the latter is not. I'll continue addressing this in response to another of your points/questions:
can you think of cases where even well matched game would lead to this outcome?
This is an interesting point that I didn't get pre-edit. But before I address it I have to start by talking about what "well matched" means. Because, as you said earlier (well, actually later, but I quoted it earlier), the most common way to evaluate skill is essentially ranked placement. With this in mind, there could be an "even matchup" but with one team playing on champs they'd take into ranked (like the kassadin of that game), where the others could be trying out new champions (like my corki and the jungle urgot).
However, I think this is another form of mismatch, and I think it's as problematic as the 'general skill level' mismatch, which is why I'm excited that the TB crew are working on a feature to adjust for this in matchmaking (it would be especially cool, although I'm not sure how it would affect que times, if TB tried to make individual lanes have even matchups).
So, with that in mind, there is only one scenario I can think of where a truly well-matched game leads to a stomp, and that's when one or more lanes happen to play better this game (when they're evenly matched in general), gain an advantage, and snowball from there. But as I said earlier, the stomp wasn't, ipso facto, the only subjective way I judged the skill matchup.
For the rest of this reply, let's presume that the teams were actually mismatched, so we can move on to the good stuff: what to do about uneven matchmaking.
How do you know that the teams were blatantly mismatched?
(especially given the following quote) I think what you're getting at here is the 'normal MMR is different than ranked MMR' thing. That is to say, that they play more ranked so their ranked and normal MMR are significantly different.
Re skill estimate: I think using current ranked placement and last seasons border is the most common approach. The problem with this is that each queue has it's own MMR so this proxy is going to have some built in error. How would you incorporate or adjust based on this?
First off, I need to say, it's absolutely imperative that ranked MMR is left out of normal matchmaking. That's because ranked MMR is indicative of your highest skill level, when you're tryharding; if it were used in normal matchmaking, you would have to tryhard in normals, or lose.
That being said, I think we want to minimize the fringe case where player X plays a ton of ranked, gets much better, then returns to normals and stomps. Therefore, here's my suggestion:
######....coming in the next post, because I've just hit the character limit on this one.
