Firebird SQL Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesFirebird SQL Development

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old November 15th, 2006, 09:45 AM
Bnunes Bnunes is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Nov 2006
Posts: 4 Bnunes User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 20 m 31 sec
Reputation Power: 0
Grouping Floats

I'm having some problems with floats atm.

I have a table where I store all material movements inside a company, where I store unit prices, quantities and discounts. I then make some reports using delphi 5.

The problem is that something is happening with the grouping / rounding of the values. Let me post an example. Imagine that the following values where entered (the exact values):

100 units of material A, unit price = 0,95, discount = 54%

Now when I make the report I do get, in most of the times, the correct values:

Quant=100; Price=0,95 €; Discount = 54%; Total=43,70 €

Unfortunately, sometimes I do get a different return:

Quant=100; Price=0,96 €; Discount = 54%; Total=44,16 €


Now, can someone give me some hint about the possible cause? Is it because of float type (hence changing it do decimal could solve it), or is the origin within delphi (masks, roundings or use of extended type)?

Thank you all in advance,

Bruno Nunes

Reply With Quote
  #2  
Old November 15th, 2006, 09:54 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 7,808 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 3 Weeks 21 h 29 m 33 sec
Reputation Power: 278
Quote:
Originally Posted by Bnunes
... or is the origin within delphi (masks, roundings or use of extended type)?
This can be easily checked by running exactly the same sql statement from the command line client.

Reply With Quote
  #3  
Old November 15th, 2006, 10:16 AM
Bnunes Bnunes is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Nov 2006
Posts: 4 Bnunes User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 20 m 31 sec
Reputation Power: 0
You're obviously correct...

Still having some trouble diagnosing it though, as it seldom returns the incorrect value... Will keep testing both the query and the report, both on a float field table and a decimal field table.

Reply With Quote
  #4  
Old November 15th, 2006, 02:38 PM
clivew clivew is offline
Contributing User
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jan 2006
Location: Carlsbad, CA
Posts: 744 clivew User rank is Sergeant (500 - 2000 Reputation Level)clivew User rank is Sergeant (500 - 2000 Reputation Level)clivew User rank is Sergeant (500 - 2000 Reputation Level)clivew User rank is Sergeant (500 - 2000 Reputation Level)clivew User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 3 Days 21 h 1 m 53 sec
Reputation Power: 19
Quote:
Originally Posted by Bnunes
You're obviously correct...

Still having some trouble diagnosing it though, as it seldom returns the incorrect value... Will keep testing both the query and the report, both on a float field table and a decimal field table.

You say you will post an example; but I do not see the SQL

Can you post it?

Typically, these problems surface when the rounding is done too early in the process.

Illustration:
Wrong: Sum(Round(Value))
Right: Round(Sum(Value))

Clive.

Reply With Quote
Reply

Viewing: Dev Shed ForumsDatabasesFirebird SQL Development > Grouping Floats


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 4 hosted by Hostway