It would make sense to primary batteries, to the point of being the overwhelmingly obvious choice. However, they aren't even using plain lithium-ion:
"Rutherford adopts an entirely new propulsion cycle, making use of brushless DC motors and high performance Lithium Polymer batteries to drive its turbo pumps."
Lithium polymer batteries being a form of lithium ion batteries that have an electrolyte with a bunch of added gelling additives, or an actual polymer electrolyte, trading some of their capacity for flexibility in form factor and leak-proofness that makes them better for things like cell phones. I don't know why they'd choose these batteries, but it's what their website says they're using.