November 19th, 2013, 12:30 PM
-
Value Too Great for Base Error, Explanation and Fix Needed
Hey Friends, its me again!
I was asked to create a script that would go into our backup directories and delete/purge anything in the directory after a certain amount of days, normally I would be able to write something up that goes to the directory finds it and deletes it.
Code:
cd /bb/backup/svn
find *.gz -type f -a -mtime +8 -exec rm {} \;
But because we suffer from a annoying time drift since we use AWS for these particular servers. using find ended up being problematic. So i wrote the below script that pretty much took the time stamp out the file name and then determined to delete or not.
Code:
#!/bin/bash
source /bb/infrastructure/generic/utils/etc/bkupperm.conf
old_file()
{
local dtcmp=`date -d "$1" +"%F"`; shift
local today=`date -d "$*" +"%F"`
return `test $((today - dtcmp)) -lt 0`
}
#Main Logic
cd $ENVR/database
for filename in *;
do
dt_file=`echo $filename | grep -o -E '[12][0-9]{3}(-[0-9]{2}){2}'`
if old_file "$dt_file" -7 days; then
rm $filename
fi
done
cd $ENVR/svn
for filename in *;
do
dt_file=`echo $filename | grep -o -E '[12][0-9]{3}(-[0-9]{2}){2}'`
if old_file "$dt_file" -7 days; then
rm $filename
fi
done
cd $ENVR/dpkg
for filename in *;
do
dt_file=`echo $filename | grep -o -E '[12][0-9]{3}(-[0-9]{2}){2}'`
if old_file "$dt_file" -7 days; then
rm $filename
fi
done
it seems to run fine but, i noticed the below error only for these two values, and i don’t necessarily understand what's wrong or what i did wrong for starters, and two how to correct it and fix it.
Error:
Code:
./delete-bkup: line 9: 2013-11-08: value too great for base (error token is "08")
./delete-bkup: line 9: 2013-11-09: value too great for base (error token is "09")
any input would be GREATLY appreciated.
thanks
November 19th, 2013, 02:23 PM
-
Wouldn't it be better to fix the time drift problem? I know at my job we had problems with time on our AWS boxes, but all we had to was set NTP up properly.
Code:
local dtcmp=`date -d "$1" +"%F"`; shift
local today=`date -d "$*" +"%F"`
return `test $((today - dtcmp)) -lt 0`
You're trying to subtract like "2013-11-19 - 2013-11-08" which is math; zero-prefixed numbers are supposedly octal but 08 and 09 aren't valid octal numbers.
You want to know if one is larger than the other, right?
Code:
local dtcmp=`date -d "$1" +"%s"`; shift
local today=`date -d "$*" +"%s"`
return `test $dtcmp -lt $today`
Also I think you had your math wrong: if a file $dtcmp is older than a date $today then $today-$dtcmp>0 (which is what I have reflected in the version above).
November 19th, 2013, 03:48 PM
-
Originally Posted by requinix
Wouldn't it be better to fix the time drift problem? I know at my job we had problems with time on our AWS boxes, but all we had to was set NTP up properly.
Code:
local dtcmp=`date -d "$1" +"%F"`; shift
local today=`date -d "$*" +"%F"`
return `test $((today - dtcmp)) -lt 0`
You're trying to subtract like "2013-11-19 - 2013-11-08" which is math; zero-prefixed numbers are supposedly octal but 08 and 09 aren't valid octal numbers.
You want to know if one is larger than the other, right?
Code:
local dtcmp=`date -d "$1" +"%s"`; shift
local today=`date -d "$*" +"%s"`
return `test $dtcmp -lt $today`
Also I think you had your math wrong: if a file $dtcmp is older than a date $today then $today-$dtcmp>0 (which is what I have reflected in the version above).
i am but a mere lacky presently, but yes. if there is a way to fix the time drift, i want to suggest it and see what is responded, because that would of surely made my life easier.
and i dont want to know if one is larger than the other. i wanna subract the amount of days between today date and the files date and subtract. if its a negative number, below 0 it deletes because its past the 7 day set, if its a positive number it isnt time to be deleted yet. thats why the math is like that when i run it.
November 19th, 2013, 04:34 PM
-
Originally Posted by gkelly1117
and i dont want to know if one is larger than the other. i wanna subract the amount of days between today date and the files date and subtract. if its a negative number, below 0 it deletes because its past the 7 day set, if its a positive number it isnt time to be deleted yet. thats why the math is like that when i run it.
Tomato, tomato. You say that 10-7>0 and I say that 10>7, it's the same either way.
November 19th, 2013, 06:08 PM
-
Originally Posted by requinix
Tomato, tomato. You say that 10-7>0 and I say that 10>7, it's the same either way.
well when you put it that way you are right,
however, still have my issue with the values of 08 and 09
how can i convert numbers to base10?
bash takes numerical input with leading zeroes to be octal values, hence 8 & 9 are too great it seems.
November 19th, 2013, 06:15 PM
-
The biggest change in what I posted was using a Unix timestamp for the time (%s) instead of at Y-m-d string (%F). Because those are actual numbers that you can do math with.
November 19th, 2013, 06:26 PM
-
Originally Posted by requinix
The biggest change in what I posted was using a Unix timestamp for the time (%s) instead of at Y-m-d string (%F). Because those are actual numbers that you can do math with.
i just noticed you did that, ok. that is part of the problem, i started off using %s, my boss, is the one that wanted the date stamps with %F so thats why I am stuck in this situation
November 19th, 2013, 07:29 PM
-
We're both talking about the same section of code, right? The part where it does the arithmetic to decide whether a file is old?
We are?
Your boss is wrong. Subtracting dates using Y-m-d format is wrong. It will not work.
November 19th, 2013, 07:36 PM
-
Originally Posted by requinix
We're both talking about the same section of code, right? The part where it does the arithmetic to decide whether a file is old?
We are?
Your boss is wrong. Subtracting dates using Y-m-d format is wrong. It will not work.
I am almost certain we are talking about the same section of code.
I ended up doing the below, because i was told converting to base10 would fix it.
Code:
old_file()
{
local_dtcmp="10#$(date -d "$1" +"%F")"; shift
local_today="10#$(date -d "$*" +"%F")"
return `test $((local_today - local_dtcmp)) -lt 0`
}
still no luck.
a friend of mine suggested that with the GNU Core Utils installed I could do something like
Code:
date --date="7 days ago"
know any truth to that?
November 19th, 2013, 08:12 PM
-
Converting to base 10 will not fix it. I'm sorry but your boss is wrong. You're going to have to find some way to either circumvent him or show him why his version is not working.
%F will give Y-m-d format. That's 2013-11-19 for today. Treating that like a mathematical expression results in "2013 minus 11 minus 19" which is 1983. Not the year but the number one thousand, nine hundred, eighty-three.
Now if the file was from 2013-11-08 you would have a grand total of
Code:
2013-11-19 - 2013-11-08
which is -49 (negative 49), and then minus "08" which is either eight if you think decimal or an invalid number if you think octal. The system thinks octal because the value "08" starts with a zero, thus the number is invalid, thus the error.
But being octal doesn't matter because the entire idea is ridiculous to begin with.
The --date thing is just an alternative syntax to the -d "$1" you have now. It won't make a difference.
November 20th, 2013, 07:57 AM
-
Seconded, your boss is wrong. "2013-11-20" is a string. Using it in math will get you different results in different languages, none of them what you (or your boss) is expecting.
HEY! YOU! Read the New User Guide and Forum Rules
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin
"The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002
Think we're being rude? Maybe you
asked a bad question or you're a
Help Vampire. Trying to argue intelligently? Please
read this.
November 20th, 2013, 01:57 PM
-
Originally Posted by gkelly1117
. . .
I ended up doing the below, because i was told converting to base10 would fix it.
. . . E t c . . .
still no luck.
Forget your friend et.al.
Try this:
Code:
old_file()
{
(( m7ss=( `date +%s` - 7 * 86400 ) ))
oldt=`date -d"$1" +%s`
(( d = (m7ss - oldt)/86400 ))
return $d
}
November 20th, 2013, 03:24 PM
-
Originally Posted by LKBrwn_DBA
...

People still use these things?
November 21st, 2013, 07:53 AM
-
Dude I remember those things. Do you think we can be part of his link exchange or web ring?
HEY! YOU! Read the New User Guide and Forum Rules
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin
"The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002
Think we're being rude? Maybe you
asked a bad question or you're a
Help Vampire. Trying to argue intelligently? Please
read this.