It'll depend; some abilities will fizzle out, some will attempt to hit their target if possible. Do you have any specific abilities or examples?
For example, on projectiles,
Caitlyn's ult can be negated by a well-timed Fizz trollpole.
To my knowledge, it can also follow teleports and recalls; if she fires the shot but they teleport or recall before they are struck, the projectile will continue to follow them (this may need testing, as they have just updated her ult very recently; but in the past, this had been the case).
If we're looking at unit-targetted dashes vs untargetable states, let's look at a unit-targetted dash that also makes you untargettable:
Yi's Alpha Strike can be negated by a well-timed Fizz trollpole.
He will appear at the cast location if Fizz was the initial target, if Fizz trollpoles his Alpha Strike.
If he Alpha Strikes someone as they recall or teleport, it's possible to have the damage hit as they move; Master Yi will appear at the cast location, and not the new location of the enemy. However, if they flash or move a smaller distance, he will be "carried" to the initial target's location.
Alpha Strike normally cannot hit enemies that he cannot see (in fog of war or stealth).
Yi can also use Alpha Strike to dodge many skillshots, and certain (damage) portions of abilities such as Karthus's R.