Why is it when 3 inhibs are down and 3 allies arent helping push lanes I'M at fault?

If I decide "screw it, if they won't help push, I won't push" because 3 of them are always walking around jg looking for kills while THREE inhibs are down and I can't get rid of them with only 1 other player's help this far into the game why do people pin it on me. This seems to consistently happen so I've just started going into the jungle and farming so the game ends. . Am I reportably "afk" to farm jungle instead of defending if they've been doing the same thing for the last third of the game? I really don't see the reasoning. I haven't been punished for it yet but I'm paranoid enough that it bugs me. Is the last 1-2 minutes of the game so sacred? What is the difference between what I'm doing and dying while trying to beat 6+ waves of super minions and the enemy team with only 2 people? We lose either way... It's like they have the "we're going to make a comeback" mentallity but don't actually want to put in the work of clearing out minions because that's BORING