No, it's you being an idiot thinking that if you miss so much as a single minion that you'll lose everything. Jungler came and ganked, but you barely lived through it, RECALL.
This is where things get complicated: Your jungler is pushing that wave out so that the enemy misses experience, money, and when you do get back WITHOUT BURNING TP, the wave will be equalized. You'd rather let the turret get all your experience, instead of that fifth person on your team, who was so kind as to actually gank the "all-game farm-fest" that is top lane?
If you aren't in lane, it isn't taxing. It's helping you out.