#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2012
    Location
    Baltimore, MD
    Posts
    27
    Rep Power
    0

    Value Too Great for Base Error, Explanation and Fix Needed


    Hey Friends, its me again!

    I was asked to create a script that would go into our backup directories and delete/purge anything in the directory after a certain amount of days, normally I would be able to write something up that goes to the directory finds it and deletes it.

    Code:
    cd /bb/backup/svn
    
    find *.gz -type f -a -mtime +8 -exec rm {} \;
    But because we suffer from a annoying time drift since we use AWS for these particular servers. using find ended up being problematic. So i wrote the below script that pretty much took the time stamp out the file name and then determined to delete or not.

    Code:
    #!/bin/bash
    
    source /bb/infrastructure/generic/utils/etc/bkupperm.conf
    
    old_file()
    {
        local dtcmp=`date -d "$1" +"%F"`; shift
        local today=`date -d "$*" +"%F"`
        return `test $((today - dtcmp)) -lt 0`
    }
    
    #Main Logic
    
    cd $ENVR/database
    
    for filename in *;
    do
        dt_file=`echo $filename | grep -o -E '[12][0-9]{3}(-[0-9]{2}){2}'`
        if old_file "$dt_file" -7 days; then
            rm $filename
        fi
    done
    
    cd $ENVR/svn
    
    for filename in *;
    do
        dt_file=`echo $filename | grep -o -E '[12][0-9]{3}(-[0-9]{2}){2}'`
        if old_file "$dt_file" -7 days; then
            rm $filename
        fi
    done
    
    cd $ENVR/dpkg
    
    for filename in *;
    do
        dt_file=`echo $filename | grep -o -E '[12][0-9]{3}(-[0-9]{2}){2}'`
        if old_file "$dt_file" -7 days; then
            rm $filename
        fi
    
    done
    it seems to run fine but, i noticed the below error only for these two values, and i donít necessarily understand what's wrong or what i did wrong for starters, and two how to correct it and fix it.

    Error:

    Code:
    ./delete-bkup: line 9: 2013-11-08: value too great for base (error token is "08")
    ./delete-bkup: line 9: 2013-11-09: value too great for base (error token is "09")
    any input would be GREATLY appreciated.

    thanks
  2. #2
  3. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,756
    Rep Power
    9397
    Wouldn't it be better to fix the time drift problem? I know at my job we had problems with time on our AWS boxes, but all we had to was set NTP up properly.

    Code:
    local dtcmp=`date -d "$1" +"%F"`; shift
    local today=`date -d "$*" +"%F"`
    return `test $((today - dtcmp)) -lt 0`
    You're trying to subtract like "2013-11-19 - 2013-11-08" which is math; zero-prefixed numbers are supposedly octal but 08 and 09 aren't valid octal numbers.

    You want to know if one is larger than the other, right?
    Code:
    local dtcmp=`date -d "$1" +"%s"`; shift
    local today=`date -d "$*" +"%s"`
    return `test $dtcmp -lt $today`
    Also I think you had your math wrong: if a file $dtcmp is older than a date $today then $today-$dtcmp>0 (which is what I have reflected in the version above).
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2012
    Location
    Baltimore, MD
    Posts
    27
    Rep Power
    0
    Originally Posted by requinix
    Wouldn't it be better to fix the time drift problem? I know at my job we had problems with time on our AWS boxes, but all we had to was set NTP up properly.

    Code:
    local dtcmp=`date -d "$1" +"%F"`; shift
    local today=`date -d "$*" +"%F"`
    return `test $((today - dtcmp)) -lt 0`
    You're trying to subtract like "2013-11-19 - 2013-11-08" which is math; zero-prefixed numbers are supposedly octal but 08 and 09 aren't valid octal numbers.

    You want to know if one is larger than the other, right?
    Code:
    local dtcmp=`date -d "$1" +"%s"`; shift
    local today=`date -d "$*" +"%s"`
    return `test $dtcmp -lt $today`
    Also I think you had your math wrong: if a file $dtcmp is older than a date $today then $today-$dtcmp>0 (which is what I have reflected in the version above).
    i am but a mere lacky presently, but yes. if there is a way to fix the time drift, i want to suggest it and see what is responded, because that would of surely made my life easier.

    and i dont want to know if one is larger than the other. i wanna subract the amount of days between today date and the files date and subtract. if its a negative number, below 0 it deletes because its past the 7 day set, if its a positive number it isnt time to be deleted yet. thats why the math is like that when i run it.
  6. #4
  7. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,756
    Rep Power
    9397
    Originally Posted by gkelly1117
    and i dont want to know if one is larger than the other. i wanna subract the amount of days between today date and the files date and subtract. if its a negative number, below 0 it deletes because its past the 7 day set, if its a positive number it isnt time to be deleted yet. thats why the math is like that when i run it.
    Tomato, tomato. You say that 10-7>0 and I say that 10>7, it's the same either way.
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2012
    Location
    Baltimore, MD
    Posts
    27
    Rep Power
    0
    Originally Posted by requinix
    Tomato, tomato. You say that 10-7>0 and I say that 10>7, it's the same either way.

    well when you put it that way you are right,

    however, still have my issue with the values of 08 and 09

    how can i convert numbers to base10?

    bash takes numerical input with leading zeroes to be octal values, hence 8 & 9 are too great it seems.
  10. #6
  11. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,756
    Rep Power
    9397
    The biggest change in what I posted was using a Unix timestamp for the time (%s) instead of at Y-m-d string (%F). Because those are actual numbers that you can do math with.
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2012
    Location
    Baltimore, MD
    Posts
    27
    Rep Power
    0
    Originally Posted by requinix
    The biggest change in what I posted was using a Unix timestamp for the time (%s) instead of at Y-m-d string (%F). Because those are actual numbers that you can do math with.
    i just noticed you did that, ok. that is part of the problem, i started off using %s, my boss, is the one that wanted the date stamps with %F so thats why I am stuck in this situation
  14. #8
  15. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,756
    Rep Power
    9397
    We're both talking about the same section of code, right? The part where it does the arithmetic to decide whether a file is old?

    We are?

    Your boss is wrong. Subtracting dates using Y-m-d format is wrong. It will not work.
  16. #9
  17. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2012
    Location
    Baltimore, MD
    Posts
    27
    Rep Power
    0
    Originally Posted by requinix
    We're both talking about the same section of code, right? The part where it does the arithmetic to decide whether a file is old?

    We are?

    Your boss is wrong. Subtracting dates using Y-m-d format is wrong. It will not work.
    I am almost certain we are talking about the same section of code.

    I ended up doing the below, because i was told converting to base10 would fix it.

    Code:
    old_file()
    {
        local_dtcmp="10#$(date -d "$1" +"%F")"; shift
        local_today="10#$(date -d "$*" +"%F")"
        return `test $((local_today - local_dtcmp)) -lt 0`
    }

    still no luck.

    a friend of mine suggested that with the GNU Core Utils installed I could do something like
    Code:
    date --date="7 days ago"
    know any truth to that?
  18. #10
  19. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,756
    Rep Power
    9397
    Converting to base 10 will not fix it. I'm sorry but your boss is wrong. You're going to have to find some way to either circumvent him or show him why his version is not working.

    %F will give Y-m-d format. That's 2013-11-19 for today. Treating that like a mathematical expression results in "2013 minus 11 minus 19" which is 1983. Not the year but the number one thousand, nine hundred, eighty-three.

    Now if the file was from 2013-11-08 you would have a grand total of
    Code:
    2013-11-19 - 2013-11-08
    which is -49 (negative 49), and then minus "08" which is either eight if you think decimal or an invalid number if you think octal. The system thinks octal because the value "08" starts with a zero, thus the number is invalid, thus the error.

    But being octal doesn't matter because the entire idea is ridiculous to begin with.

    The --date thing is just an alternative syntax to the -d "$1" you have now. It won't make a difference.
  20. #11
  21. Sarcky
    Devshed Supreme Being (6500+ posts)

    Join Date
    Oct 2006
    Location
    Pennsylvania, USA
    Posts
    10,696
    Rep Power
    6351
    Seconded, your boss is wrong. "2013-11-20" is a string. Using it in math will get you different results in different languages, none of them what you (or your boss) is expecting.
    HEY! YOU! Read the New User Guide and Forum Rules

    "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin

    "The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002

    Think we're being rude? Maybe you asked a bad question or you're a Help Vampire. Trying to argue intelligently? Please read this.
  22. #12
  23. Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Sep 2006
    Posts
    834
    Rep Power
    387

    Cool


    Originally Posted by gkelly1117
    . . .
    I ended up doing the below, because i was told converting to base10 would fix it.

    . . . E t c . . .
    still no luck.
    Forget your friend et.al.
    Try this:
    Code:
    old_file()
    {
      (( m7ss=( `date +%s` - 7 * 86400 ) ))
      oldt=`date -d"$1" +%s`
      (( d = (m7ss - oldt)/86400 ))
      return $d
    }
  24. #13
  25. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,756
    Rep Power
    9397
    Originally Posted by LKBrwn_DBA
    ...


    People still use these things?
  26. #14
  27. Sarcky
    Devshed Supreme Being (6500+ posts)

    Join Date
    Oct 2006
    Location
    Pennsylvania, USA
    Posts
    10,696
    Rep Power
    6351
    Dude I remember those things. Do you think we can be part of his link exchange or web ring?
    HEY! YOU! Read the New User Guide and Forum Rules

    "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin

    "The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002

    Think we're being rude? Maybe you asked a bad question or you're a Help Vampire. Trying to argue intelligently? Please read this.

IMN logo majestic logo threadwatch logo seochat tools logo