Well, yes, even the sun will run out one day - but I hope we as a species will have taken appropriate steps well before that happens.
Seriously, yes, even fissile material is finite, but it's a step in the right direction.
I use lead-acid batteries, 1320ah of them, and I'm off-grid, so I don't know very much about grid-tie systems and the issues they raise. I'm just saying it's possible to live with batteries, and there are even some advantages. They do need periodic replacement (every 8-9 years in my case), but much of it is recycleable, so it isn't just dumped in a landfill. I believe the price of lead in the last few years makes it much more attractive to recycle lead-acid batteries.