Hello,
I suggest Python (http://www.python.org).
It's cross platform, has good UI support (I recommend wxPython, http://www.wxpython.org/) and is easily extendable with
c/c++ modules. (See http://www.boost.org/libs/python/doc/index.html for example).
Pluse it has a very good mathematical library (http://numpy.sourceforge.net/) which is cross platform as well.
Write most of the logic in Python and only the time critical section in c/c++ (ANSI C will be best cross platform solution for this).
HTH.
Miki