Bait. He's dependent on his turrets more than you think for damage output since his rework last year. In the case of Yi, you are screwed IF you engage with Q.
Don't Q to him to engage, just run at him, his E is his only CC and it's not the easiest to land. If you are a moving target it's harder obviously and he needs to hit the centre for it to stun else it'll just slow you, something a Yi can handle but watch his animations, you can't be hit by the stun if he does it while you are in Alpha. Q into him when he throws it.
Other option is to get him away from his turrets, which greatly lowers his damage output and mean's he's not 'safe' anymore but that's harder as Yi, you need a poker to do that.