Perl. When things get to messy for a spreadsheet, I whip up a little perl. Easier to repeat the calculations for different data sets, as a bonus. Access to much richer libraries, and you can shell out to GNUPlot, Ploticus, Asymptote, or whatever.
Or Python, if that's your cup of tea ... or Ruby, even R ... whatever scripting language floats your boat.