|
|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
#1
|
|||
|
|||
|
Numerical Precision
Hi. I need your assistance in this one:
I'm importing a big Text file with currency quantities in it to a FB table via EXTERNAL FILE. I do it in a two step process, because the original file I recieve is character sepparated with "|", so I have to transform it to a fixed lenght. I do that with a little Delphi program I wrote. The problem arises in how it is tranformed, because I can't get the same result in FB when I do a SUM of all the records. I made the Delphi program to Sum every record it processed (5.4 million) and I get one number. If I do a SUM on the external table in FB I get another one (Sure, a difference of about half a million in a 86.7 billion total is less than 1%, but half a million can't do). What should I check? |
|
#2
|
||||
|
||||
|
Quote:
You should check the datatype used for storing the values ! |
|
#3
|
|||
|
|||
|
Quote:
I used Format( "%15.2f", [Var]) to write down the values in the text file, defined the field as CHAR of 15 in FB and without transferring nothing to an internal table I run a SELECT... SUM( CAST( Field AS DECIMAL(15,2)) and SUM( CAST( Field AS DOUBLE PRECISION) obtaining the same result between them, but different from what Delphi totals. |
![]() |
| Viewing: Dev Shed Forums > Databases > Firebird SQL Development > Numerical Precision |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|