Might force everyone to keep checking what new spells got swapped every time enemies come back into the lane, even if its not the lane your in. You have to know what they bring to the fight in case they come roaming your way. This would be especially annoying for bot lane where the clown fiesta usually takes place. You'd have to account for the swapped spells in whatever fight you got into before the fight takes place. And can you imagine the clown fiesta if everyone could at any point just get teleport and come bot, that ought to be hilarious and frustrating.
But you'd also have to take into account your enemy having all possible relevant spells at the same time anytime you don't have vision of them, because you just don't know what they have until you see them, maybe they switched barrier for exhaust or ignite or vise versa, maybe they got ghost now and can run you down. This can get rough when teamfights happen as every new fight everyones spells could have changed, at least within cooldown limitations of course.
You can also end up with a lot of sudden teleports at any point in the game or whole teams with both ghost and flash once the early game is over and they no longer need whatever spells they needed at the start of the game for the edge. Might be good for teams that get ahead early and want to force teamfights, get everyone ghost and flash and lets see you try to disengage from that. Or maybe that will be a late game thing. Or if you think the healing is bad now, imagine a entire team getting the heal spell, that ought to be fun.
Right now you don't have to worry about any of this, its just a question of does the enemy have his spells of not, not what they are because they are set in stone and in full view as early as the loading screen.
EDIT: Or what if we get a new kind of clown fiesta, this time it can happen top or mid or just about anywhere because now bot lane gets to join in with the fun.