Player A has 150 victory units. They use all of them in ranked, flex, normal, and ARAM. Eventually they are matched with everyone else who contributes 150 victory units in those queues.
Player B has 1000 victory units. They use all of them in ranked, but average just 150 in flex, normal, and ARAM. Eventually they are matched with people who contribute 1000 skill units in ranked, but they are matched with people who contribute 150 in flex, normal, and ARAM.
Player C has 150 victory units. They don't care much about any game mode and contribute just 100 victory units on average in every queue. They are matched with other players who contribute 100 victory units.
If Player B played seriously in those other queues, they would win some games, their MMR would rise, and they'd be matched with better players that no longer include Player A.
If Player C played seriously in all those queues, they would win some games, their MMR would rise, and they'd be matched with better players that now include Player A.
Now instead of Player A playing with people who contribute on average 150 victory units...they now play with people who contribute on average 150 victory units.