Kayn does not deal any damage while he is "inside you", and when he leaves the damage he can do is strictly dependent on being able to stay next to you.
As Vayne, you can Condemn him away from you, and since you have the majority of control over where Kayn will emerge, you should be able to set him up for an easy wall-pin-stun. Consider also taking advantage of your ultimate's invisibility during Tumble to give yourself the breathing room necessary to aim your abilities properly.
In a teamfight scenario, it may be more valuable to simply keep fighting and deal with the problem later. Once again, as Vayne, you have all the tools you need to keep yourself safe, or at the very least delay any significant damage until an ally is able to assist you. Tumble management is key here, and makes the difference between life and death.