#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Oct 2004
    Location
    Guadalajara, Jalisco, México
    Posts
    62
    Rep Power
    11

    Precision of Numeric types in Delphi


    Hi, I'm relatively new to Delphi, so be patient.

    I'm processing a text file to feed a Firebird database. The text file contains 2 fields with monetary quantities, unformated separated by "|"'s.

    I already finished the routine that parses the file and generates one with fixed lenght records to feed FB. My problem arises as I added a variable to sum the quantities along with parsing the file, and I can't get the same total as FB does. I'm using a double variable to hold the running sum, and using the Val procedure to convert from string to numeric inside Delphi. I already tested with a currency var and StrToCurr. In Delphi I get 86,710,857,594.49 and 86,710,857,594.39 with the currency var. In FB I get 86,711,048,279.58 as a decimal(15,2) sum, and 86,711,048,279.485 as a double precision. I assume Delphi is correct, but then how can I reproduce the right numbers in FB?

    The source file contains 5.5 million records, so a 1 cent per record deviation produces a 500 thousand deviation on the end. Any sugestion?
    Last edited by HereSomeHow; November 3rd, 2004 at 07:38 PM. Reason: Correction of question
  2. #2
  3. Banned ;)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Nov 2001
    Location
    Woodland Hills, Los Angeles County, California, USA
    Posts
    9,648
    Rep Power
    4248
    Try using the Extended type and see what you get.
    Up the Irons
    What Would Jimi Do? Smash amps. Burn guitar. Take the groupies home.
    "Death Before Dishonour, my Friends!!" - Bruce D ickinson, Iron Maiden Aug 20, 2005 @ OzzFest
    Down with Sharon Osbourne

    "I wouldn't hire a butcher to fix my car. I also wouldn't hire a marketing firm to build my website." - Nilpo

IMN logo majestic logo threadwatch logo seochat tools logo