Why does this happen

So Im a jungle main and im starting to fall behind after the 20 minute mark because adc and other laners like to roam around taking my jungle camps which happen to be the only safe farm available. So how am i as a jungler supposed to get caught up when my laners run around taking all the camps and then backing. I understand if a fed adc wants red and thats not an issue but why does the fed adc need my raptors and krugs too after they just took my blue buff gromp and wolves. They don't. I also understand if a fed mage wants blue and usually i can afford to give that to them. But If your jungler is far behind take your lane minions and maybe the buff if youre ahead but leave the lesser camps alone FFS or how the hell do you expect us to stay relevant.