It’s extremely difficult to remove the cells from bottom cover so yes, replacing the whole battery/bottom cover assembly is the way to go. You can find cheap parts on eBay etc.
open