Yep. That's definitely the way to go for this, but it's not without a few challenges, not least of which will be getting everyone to agree on a common format for the battery pack, fitting brackets, connector(s), and management interface. Also, since we're potentially talking a several 100kg of cells in total, that's probably not something many EV owners will want, or be able, to add and remove on at home on a regular basis, and a trip to a dealer or suitable service station before and after any longer range trips isn't ideal either. I suspect most users in that circumstance would fit all the battery packs they can and be done with it, entirely negating the benefit of not having to lug around 100s of kg of batteries they don't need, and working the cells you are using even harder to do so. Still, at least they'd be able to more easily swap them out when they deteriortate, which is at least a step up from where we are now.
Now, if you could combine this with the mooted instant EV-top up idea of simply driving into a bay and having your nearly depleted battery packs replaced with fully-charged ones in a couple of minutes, e.g. "replace my current battery pack(s) and fill another X of the remaining banks as well", then we're talking. There are a prototypes of this, and some Chinese EVs apparently use it, but everywhere else seems to be deadset on rolling out high-capacity CCS/NACS charging infrastructure instead. That's means there's a lot of investment and infrastructure that'll presumably become essentially redundant if this ever takes off, so those with their money on the line are going to fiercely oppose this idea. It'll also require another couple of shifts in mindset; to one where you don't actually own the battery packs but only lease them and pay for the charge they contain, and deciding on how a new EV is configured at the dealer, where perhaps a car comes with a built-in pack as standard, and/or just one or two removable packs fitted out of several potential slots.