#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2006
    Posts
    205
    Rep Power
    15

    Cutting down decimals


    Hi all,


    when working with decimals i know how to add the numer of decimals...i have to power it with a decimal


    like integer * 1.000 = 3 decimal number


    but now i have the inverse scenario:
    in a formula i have a lot of 6 decimal numbers and the problem is that the result is too big and gives this error:

    Arithmetic overflow or division by zero has occurred.
    arithmetic exception, numeric overflow, or string truncation.
    numeric value is out of range.
    how can i cut down decimals?


    here is the formula i use:

    UPDATE cal SET
    dif_c=IIF(COALESCE(warm_c + warmdif_c, 0) = 0, 0, ROUND(1.0 * warmdif_c / (warm_c + warmdif_c) * energy_c),6)

    where
    DIF_C, WARMDIF_C, ENERGY_C = NUMBER(15,6)
    WARM_C = NUMBER(15.2)
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2008
    Posts
    55
    Rep Power
    6

    Thumbs up


    Try see the value in use, first! If N / 0 = error of course
    if ( x < Type > x ) = error of cource

    Try to use one procedure for test.

    see:
    http://www.firebirdsql.org/refdocs/langrefupd25-psql.html
    http://www.firebirdsql.org/refdocs/langrefupd25-intfunc-round.html
    http://www.firebirdsql.org/refdocs/langrefupd25-intfunc-trunc.html
    http://www.firebirdsql.org/refdocs/langrefupd25-intfunc-coalesce.html

    or
    http://www.firebirdsql.org/refdocs/langrefupd25.html

    See the diferece between NUMERIC and DECIMAL type

    REMEMBER: ANY calculation or other with NULL, ALWAYS is NULL resulted

    n / NULL = NULL = error
    Last edited by emailx45; October 17th, 2013 at 09:52 AM.
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2012
    Posts
    19
    Rep Power
    0
    Further to what emailx45 wrote, if you just want to cut down the decimal count on a number you can cast the value as a numeric(x, y) with the y being the number of decimals.

    So for example if you

    cast(123.4567890 as numeric(5,2))

    you would get

    123.46

IMN logo majestic logo threadwatch logo seochat tools logo