When a game is remade, the "leaver penalty" is given to anyone who is marked as AFK when the remake occurs. It's not only for the player who initially leaves. There are certain behaviors that will mark you as AFK even if you don't physically disconnect from the game, and this:
We all agreed to stay in the fountain since we knew the adc was not coming back.
is one of them. Even if you purchase items, idling in the fountain waiting to remake a game will mark you as AFK, which unfortunately will give you the leaver penalty. If I were to hazard a guess, all of your teammates lost LP that match as they did the exact same thing as you. Even if you plan on remaking a match, it's important to continue playing as normal while waiting to remake the game as you never know if something unexpected will occur, such as the remake vote failing for whatever reason.
If you're scared of first blood cancelling the remake, you don't need to be. First blood will only prevent a remake if it occurs before someone leaves. However, if first blood occurs after someone has already left, then a remake will still be valid.
for more information on the /remake system, you can read this:
https://support.riotgames.com/hc/en-us/articles/208754686--Remake-FAQ