The Trouble With Rounding Floats 456
lukfil writes "We all know of floating point numbers, so much so that we reach for them each time we write code that does math. But do we ever stop to think what goes on inside that floating point unit and whether we can really trust it?"
This is not a "problem" per se (Score:3, Interesting)
You don't start addressing a problem in software just by assuming Float or Double will magically fill every need. An experienced programmer needs to have a knowledge of how to use, and how not to use, the programming tools at hand. TFA about floating point numbers is very introductory (at the end it mentions that the next article will tell us how to "avoid the problem"... I assume it will go on to cover some basic idioms.) In a way it misses the point: Floating-point rounding is not a "problem". Floats and Doubles always do their job, but you have to know what that job is! The behaviour of floating point numbers should not be a big surprise to a seasoned coder.
For example: You can't use float or double to store the numerical result of a 160-bit SHA-1 hash... you have to use the full 160 bits. (Duh, right?) So, if you use a mere 32 bits (float) or 64 bits (double) to store that number, you are going to sacrifice a lot of accuracy!
It used to be much worse. Kahan fixed it. (Score:5, Interesting)
Due to the efforts of Willam Kahan [berkeley.edu] at U.C. Berkeley, IEEE 754 floating point, which is what we have today on almost everything, is far, far better than earlier implementations.
Just for starters, IEEE floating point guarantees that, for integer values that fit in the mantissa, addition, subtraction, and multiplication will give the correct integer result. Some earlier FPUs would give results like 2+2 = 3.99999. IEEE 754 also guarantees exact equality for integer results; you're guaranteed that 6*9 == 9*6. Fixing that made spreadsheets acceptable to people who haven't studied numerical analysis.
The "not a number" feature of IEEE floating point handles annoying cases, like division by zero. Underflow is handled well. Overflow works. 80-bit floating point is supported (except on PowerPC, which broke many engineering apps when Apple went to PowerPC.)
Those of us who do serious number crunching have to deal with this all the time. It's a big deal for game physics engines, many of which have to run on the somewhat lame FPUs of game consoles.
Re:Decimal Arithmetic (Score:3, Interesting)
I bothered to ask the question of what to use for monitary
usage at a financial institution in my recent past. I was
a bit ( pardon the pun ) suprised to get a blank stare, to
have to explain what I was talking about. Floats where good
enough. Course, I had a problem in
a list of values ( testing, each was
sum wasnt 1.0. Had to do a bunch of
decimal.parse(value.ToString())
to get things to sum up correctly.
Must read floating-point articles (Score:3, Interesting)
and
When bad things happen to good numbers [petebecker.com] (as well as Becker's other floating-point columns on that same page)
average joe (Score:2, Interesting)
when i use my calculator, it doesn't give rounded off numbers. i suspect lots of programs will have problems with rounding off but i don't seem to notice it. is it that insignificant?
Why I only use decimal values (Score:4, Interesting)
Re:average joe (Score:2, Interesting)
The problem is using floating point improperly (Score:5, Interesting)
The trick is to carefully calculate exactly how much error each operation can generate. It is possible to know exactly how many bits of your result contain valid information. If you need more accuracy, you can split it into multiple operations. As long as the final accumulated error in their result is less than
Another interesting problem occurs with floating point results. You cannot expect the complete answer to be exactly identical on all machines. Even on the same machine, compiler settings affect the answer: x87 differs significantly from SSE. If you are doing something that needs bitwise identical results on all machines, you need to either implement it with integer math, or do what GIMPS does and do error tracking.
Melissa
Re:Decimal Arithmetic (Score:3, Interesting)
Re:average joe (Score:3, Interesting)
Not true.
In the math class I teach, I do the following: have everyone take a calculator and do "2/3".
Half of the calculators say this: "0.666666666" (rounded down).
Half of the calculators say this: "0.666666667" (rounded up).
In truth, an exact answer requires an infinite sequence of "6"'s. The calculator (or any computer) must decide whether to round up or down to fit it into its display space (or memory). You always have some round-off error -- and the more calculations you do, the more the round-off error builds up and up and up....
Another true story (Score:3, Interesting)
I once wrote interest-calculation software for a bank. This was new software to replace their old stuff. Naturally, I stored the values in cents, not guilders/dollars/euros, to avoid rounding errors (which really have a big effect in interest calculations).
When I delivered my software, they compared my output to the output their old software produced. There were small differences. They asked me where these came from, and I traced them back to rounding errors in their old software. I showed them this by example, thinking that they would be happy that their new software did not have this problem.
Their response? "The new software should produce exactly the same numbers as the old software." "But the old numbers are WRONG!" "That does not matter, the new software should produce exactly the same numbers as the old software."
It is really hell to make good new software error-compatible with faulty old garbage.
Discrete vs Continuious (Score:1, Interesting)
Floats are approprate when the value is a truely continuious one that cannot ever be precisely represented in a digital computer system. A classic example would be wave high computation simulations - the "true hight" of a wave being represented could be measured down increasingly better levels -- all the way to the subatomic level, there's no actual 'precisely right' definition. Thus, a float (or a double) is the correct data type. (Or an audio wave file - the actual 'real world' sound pressure is a continious value. CD players just approxmate it with 16 bit integers - and yes, there is a certain quantatisation error with that.)
But in finance, there is a "true" value - my bank account has, for example, two hundred dollars and fifty six cents - it is a completely discrete value. No matter how many times I measure it, it will be exactly the same.
Re:Decimal Arithmetic (Score:5, Interesting)
Friends of mine went off to work "In The City", when I quizzed them about their use of numbers for stock prices etc they were equally dismayed that things were being passed around as doubles. Often encoded as ASCII text in data streams as well, requiring different people to write their own ASCII->DOUBLE conversion depending on the representation of the stock tick. I think this kind of madness is quite prevelant.
As someone else pointed out, if you want to do things properly you can end up needing very big integers.
Perhaps the best option is to make sure people can only by and sell equities etc in numbers that can be exactly represented as doubles on a computer. It sounds crazy, but it's not as crazy as it looks. One of the reasons stocks etc are quoted as they are is probably due to the ease of the mental arithmetic.
Kudos to the parent of your post. At least he knows what he is having to do is dodgy and cares enough to check!
Re:Why I only use decimal values (Score:3, Interesting)
Re:Decimal Arithmetic (Score:5, Interesting)
I'd be not only dismayed but very surprised to find anything which interfaces to the London Stock Exchange passing stock prices around as doubles, or as any other kind of floating point number.
The LSE feeds all use 18 digits for values, with the first 10 being implicitly before the decimal point and the remaining eight being after the implicit decimal point. This is very handy because it means all the values can be manipulated using 64 bit integers. The LSE rules also state very precisely how rounding must be handled. If you try to submit a multi-million pound deal and your calculation of the consideration is out by just one penny then the deal will be rejected.
No-one with the slightest clue about how to code would use floating point maths in any kind of financial program, particularly not one where they're working with the LSE.
Re:Decimal Arithmetic (Score:3, Interesting)
This caused some big problems for me in a previous job. I was using something similar to server-side javascript to generate financial reports (including summing and currency conversion) which the customer was testing by trying to get the same results in Excel. I knew there was a floating point issue in my code, but even after I fixed it it didn't match the customer's Excel spreadsheet. Imagine trying to tell a customer that the problem is with Excel!
Re:Decimal Arithmetic (Score:3, Interesting)
Can you outline examples of these conformance tests, or even better, are they freely available? I assume these are intended to make sure things that go on the wire have a sane value, fall between certain daily trading limits etc (to prevent things like the Mizuho cock-up [economist.com]) [*].
I suspect that the main culprit of "dodgy doubles" is likely to be people throwing together ad hoc codes behind the scenes, not the official interface to the exchanges.. (the "front" and "back" doors I mentioned earlier).
NatW^H^Hmeless huh? That gives me some confidence. Though it still doesn't make me happy with their online banking (the numbers don't add up). I still need to find someone that can make my balance - you know - balance..... Yes, I know, "retail" banking... Yuk!
I assume you know plenty of people that work deeper in the bowls of such organisations. Would you do an unofficial survey and find out how other people implement financial numerics? I'd wager you will be shocked!
[*] Tee hee, that story did make me smile. Poor guy I bet he felt terrible....
Re:Error diffusion is another way. (Score:3, Interesting)
Lots of programmers though, are unaware of the finer details of floating-point numbers.
As evidenced by MS-Excel failing to give the correct answer, even when as we've now demonstrated, there's multiple, simple, correct algorithms for doing so.
It *is* surprising to do the equivalent of 1000 + (1+1+1 ...[10000 times] +1) and get 1000 as the answer. The answer *should* be on the order of 11000. Anything else is a bug.
Thus we can conclude that MS-Excel does not even manage to get its SUM() function correct. One of the simplest functions there can be on a set of floats.
Re:This is why you would choose... (Score:1, Interesting)
Clasic testbokk issue (Score:3, Interesting)
Why would this count as "news". Everyone who has to deal with this would already know about it.