I concur: the Python shell is a very very powerful calculator given that you can define functions in the interpreter. There are many graphics packages for Python; Matplotlib is perhaps the most complete albeit not the symplest. As suggested above, installing Python with the IPython shell, NumPy and SciPy, enables the "PyLab" IPython mode, which is similar to what Matlab would offer in terms of graphics and computation integration.

Simpler to install and learn is perhaps Octave (with plots using GnuPlot), which would behave similarly. Although for the long term, I'd say learning the Python shell is more useful than learning Octave.