I've just had my first bank go puffy (RAV Power, I think?) not all that long ago, and it looked like there were 18650s used inside already. I just haven't gotten to trying to replace them yet, so I'm not sure. Have you looked inside yours? I don't see why you can't just exchange them if so. Might require a minor bit of de/soldering work.
I remember seeing charger bank shells on eBay.
Finally, I think a full blown RPi is serious overkill for your intended purpose. At least consider a Pico - much smaller, cheaper, and actually intended for simpler uses like this (although it's still kinda overkill).
I do find your hot-swap idea interesting, why would you need to do this if you had pass-through?