First of all, even when dragon gave a stacking buff, the AD+AP %bonus buff was the first buff, giving a power boost to the team that got the first dragon which is typically the team with the strongest early game, which allowed them to secure future dragons even after the other team started to hit mid game.
Second, teams typically have at least 2-3 strong early game members even if they incorporate some scaling champions like GP or Vayne etc. It is exceedingly rare that your team is so late game focused and the enemy team is so early game focused that it is a foregone conclusion they will get the dragon. If that's the case, that is earnestly a drafting problem and on your team.
Third, dragons aren't always determined by teamfights. Early dragons can often be solo'd by a jungler, early dragons can be taken by the jungler plus 2 botlaners after a successful gank, early dragons can be stolen, and early dragons can be the result of a TP gank by the toplaner, etc.
Fourth, failing to steal a dragon is a failure of your team. It is literally getting outplayed, or more accurately failing to outplay your opponent.
Fifth, if your team is so blatantly late game oriented and the enemy team is so early game oriented, even them having 3 infernals isn't going to change that. If anything it's partly wasted on them provided they fail to close out the game fast enough.
Sixth, if your team had better positioning and vision then I can only conclude you misplayed and lost the teamfight of your own accord, you'd have to have huge drafting problems if you had all of these advantages and are insisting you lost due to raw numbers. Skill is always a factor, otherwise early game champs winning lane and late game champs losing lane for instance would be assured when your jungler is an intervening force that prevents this from being set in stone. The exact same thing applies to dragons.