VB 2010 fails at basic math???

This is frustrating. I'm doing some really, really basic stuff, and getting a ridiculous result.



Should be pretty straightforward, but I get some weird results. I had a case where TotalHours was 10.04, and RegularHours was 8. The function returned 2.0399999999999991. If I enter into my immediate window: I get the same result. Obviously, I can round the result, but I'm curious why this happens at all, so that I can hopefully avoid this in the future.

Computers cannot represent all numbers perfectly: 10.04 on yours is probably stored as 10.0399999999999991. Subtract 8 from that and you get 2.0399999999999991.

The solution is to round the result.

Notice, that even between 0 and 1 you have infinitely many numbers (e.g. 0.1, 0.01, 0.001, 0.0001, ...). It would be really wonderful world, if you could store them all in only 64 bits. Double variable store only rounding of a number.
You can round numbers, like requinix said and it should work in most cases. However not always.
For example, for checking equality of double1 and double2, you do should rather use this
where epsilon is some small number (you can think about it as "the precision of equality").

Still, floating-point numbers are pretty complex. If you want use them for something serious, you should do some research first.
(you can start here http://www.cprogramming.com/tutorial/floating_point/understanding_floating_point.html )

This is going to turn more into a rant than a request for help, so I'm sorry in advance.

One cannot expect to have to round every number to a fixed number of decimal points after every mathematical operation. While I can certainly do it if necessary, the need for it in this case is completely unjustified.

If I ask my calculator 10.04-8, it tells me 2.04, as it should. I would expect a sophisticated programming language to be as accurate.

Here is another example that I ran into with the same program:

Imagine you are writing a program that simply builds a receipt or an invoice. You write a function that accepts the selling price of an item, and how many of that item were sold.

In my case, 3.2 quarts of oil were sold, at $1.56 per quart. So my function should return 3.2 * 1.56, which according to my calculator, is 4.992. According to vb, it is 4.9920000000000009. I can't go back and round every number I've ever performed basic math on.

Again, I can type these values directly into my Immediate window and get the same results, so I can't imagine this is an error in my code.

I appreciate the information as well as the suggestions, but something is clearly wrong.

This is going to turn more into a rant than an answer, so I'm sorry in advance.

What's "clearly wrong" is your understanding of computers. And of your calculator: when it calculates 3.2*1.56 it comes to the same answer your computer does. But whoever wrote the software for it actually understood the nature of computers and took appropriate precautions against showing you an answer that's a millionth of a millionth off. Because guess what the calculator does after it gets the answer? Rounds. Your calculator shows only so many digits so it rounds to an epsilon small enough to not impact the value displayed.

So I'm sorry VB isn't the "sophisticated programming language" you expect it to be. I'm sorry the world of computing is more complicated than what you're used to seeing on a calculator. And I'm sorry that you have to go through the tremendous effort of adding a function call when you do floating-point arithmetic. You're right: something here is clearly wrong.

First off, chill the hell out. You seem to have misinterpreted my frustration as being with your response, which it was not. A personal attack is simply not cool.

I've never worked with a programming language that made me handle rounding every bit of (floating point) arithmetic manually before. Therein lies my frustration.

My initial question was why this occurs, not how I am supposed to fix it. That was explained nicely by number47, and his link was helpful. My second post was me expressing further frustration that VB doesn't handle this rounding itself, as every other language I've worked with does.

Duh. I don't expect much from VB, but compare it to a calculator, and tell me which one has more sophistication. If I had a choice in which language to use, VB would not be it.

And I'm sorry for the bug that crawled into your *** and died.

I don't write the coding standards here, so I have no say in this. Ever hear of code review? With that said, no language I've ever worked with has required this, hence my confusion and frustration.

So yeah, thank you Number47 for being helpful, and requinix thanks for the laugh at your misunderstanding :-)

I would like to comment on the fact that the OP said none of the previous languages exposed this issue to him before.

It is not uncommon that languages have internal default settings for the display of real numbers. It takes an explicit format to extend it past the default.

This default issue has wider consequences. What else do we not know about the behavior of our programming languages.

I discovered years ago the compiler options that have a primary purpose sometimes a secondary effect. In my case turning on an optimization gave different results. No code change, just that the two executable programs ran differently with the different optimization setting.

Woow, slow down guys. (btw how do you spell woow in English?)
macrado, you complained at something which is natural for computer beings, and it has looked like demanding of something more. You know, it can be a little bit irritating. Thousands of computer scientists try hard to find the best solution, and someone comes and says "Oh, it's so stupid, they should do something more".

admiraln, the optimization can change the order of evaluation, so if something depends on the order the result can be different. But, if it is another matter it can be a bug in the compiler. Or optimization has some code requirements ("you must use this", "you mustn't use this", ...).

Oh, and sorry for smarassing, but you know, I'm so wonderful and everything, so ... I couldn't resist.

I'm a girl. A pat and one pack of puffs are owed ^-^

Sorry for all of my frustration guys. We just switched over to VB 2010 for several of our projects here, and it has been frustrating trying to update our code. On top of that, management has suddenly become extremely nitpicky about our coding standards (No using this function, no using that function, variable names will follow this specific format, etc). Finding that some basic operations were not returning the results I expected just pushed me over the edge, I guess.

As I mentioned, other languages I've worked with, including earlier versions of vb, have always done the rounding on floating point variables themselves, so I was freaking out about what was going on. The closest I've come to this is some rounding issues while storing single point floats into a database.

I really do appreciate the help that was given even if some of it had to come in the form of some tough love.

Nonsense. Everyone knows there are no girls on the internet. Thank you again for your help!

It's called IEE 754 people, and pretty much every programming language uses it. I'm having a hard time thinking of one that doesn't.

That's what's kind of silly here: why call out VB.Net on this, when nothing else you try will be any different in spite of what you say about prior languages doing this for you?

I am curious does the fact CPU hardware or a software library use iEEE 754 force a programming language to display a certain precision. I suspect not. The under lying hardware my be running double precision but the default display is say only 10 significant digits.
In this case you would not see the details of the implementation.

For what you're expecting, I think you really wanted to use "decimal" instead of double. Decimal will add and subtract in the manner you expect.