|
|||||||||
|
|||||||||
| |||||||||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
Dev Shed Forums Sponsor:
|
|
Stop making mediocre tutorials.The best tutorials are video! Camtasia Studio makes it easy to create engaging, buzz-building screen videos at any size, in any popular format. Download the free trial!
|
|
#1
September 24th, 2004, 05:54 AM
|
|||
|
|||
|
Building a 'batch job' for crontab
Complete noob here to *NIX shell scripts and crontab. I understand how to schedule crontab from my website CPanel crontab tools.
I want to run a script that:
Can I build a file with the steps above and have crontab run that? What would the file extension need to be? Does it need to be in a specific path to run it? I know the command line ermmm... commands for rename, mysqldump and gzip, but can people point me at tutorials for doing redirects and ftp at the command line please? Thanks in advance.
|
|
#2
September 24th, 2004, 09:29 AM
|
|||
|
|||
|
Hiya,
1) You don't need to use any specific filename extension for a script to be run, from the command line or from cron. Just type the full name of the file, and use filename extensions that have meaning or everyone uses as standard. Most people use ".sh" for shell scripts ... I use ".ksh" for shell scripts written in the korn shell to make it obvious to others. 2) When you execute the script from cron put the absolute path to the script (I've never tried not putting it in, I don't know if it would work without it...) 3) Make sure you have your UNIX environment setup within the script itself since cron does not pick up the environment as you do when you login. If you don't then you could find that you can successfully test your script from the command line but it doesn't work from cron (or even worse - doesn't work the same!) (see "CRON job won't run" post by sleepyfrog below). 4) Some FTP code (not tested): (see "How to trap ftp success/failure" post by AVS below) Code:
#-- Function doftp
doftp()
{
ftp -in $host << EOF_MYFTP
user $user $password
put YOURFTP.FILE $dest_dir/$dest_file
dir $dest_dir/$dest_file
bye
EOF_MYFTP
}
#-- Run ftp
doftp 1>doftp.log 2>&1
#-- Check logfile for "No such file or directory"
NUM_ERRRORS=`grep -c "No such file or directory" doftp.log`
if [[ ${NUM_ERRORS} -eq 0 ]]; then
echo "File(s) transferred successfully"
else
echo "Errors transferring file(s):"
grep "No such file or directory" doftp.log |awk '{print $1}'
fi
5) Some email code I've used: Code:
mailx -s "WARNING" me@hotmail.com <<+++END+++ The following message has appeared on server `hostname`: `cat /var/tmp/message` +++END+++ 6) I don't think you can add additional files to a gzip file... 7) Specify the shell that the script should be run in at the top of the script using the "hash bang" command. e.g. for a korn shell script with ksh existing in /usr/bin on your system the first line in your script would be Code:
#!/usr/bin/ksh 8) Tip1: Create a header for the script 9) Tip2: Write out your comments first If you use a #-- at the start of the comments then they stand out, and you can get the "program flow" later on by executing: Code:
grep "#-- " scriptname 10) Tip3: Echo the date and time prior to running a command. This helps you develope the script, and support it when it is being used in anger. 11) Tip4: Send the output to a logfile With these tips you might start your script as below: Code:
#!/usr/bin/ksh
#################################
# Filename: backup_webdb.ksh
# Description: Backup website database
#
# Revision history:
# 0.1 rc2dotcom Created
#################################
#-- Set your environment as cron won't pick it up
# some_code
# (dangerous if your environment changes and this code doesn't)
# or
# . ~/.profile
# (better, but note it's korn shell specific)
# or
# . some_file_that_sets_the_environment
# (better still, especially if this file is "dotted" when you
# login and sets your environment at the command line)
#-- Set variables
DATETIME=`date +"%Y%m%d.%H%M%S`
LOGFILE=`basename $0.log.${DATETIME}
#-- Send stdout and stderr to a logfile
echo Sending output to logfile ${LOGFILE}
exec 1>${LOGFILE} 2>&1
#-- Take the site offline during the backup
date +"%Y/%m/%d %H:%M:%S Take the site offline during the backup"
#-- Perform a mysqldump backup
date +"%Y/%m/%d %H:%M:%S Perform a mysqldump backup"
#-- Gzip the dumpfile
date +"%Y/%m/%d %H:%M:%S Gzip the dumpfile"
#-- Adds further files to the gz file (not sure this can be done)
date +"%Y/%m/%d %H:%M:%S Adds further files to the gz file"
#-- FTP the gz file to another server
date +"%Y/%m/%d %H:%M:%S FTP the gz file to another server"
#-- Email confirmation
date +"%Y/%m/%d %H:%M:%S Email confirmation"
exit
Good luck! Andy Last edited by andyb1ack : September 24th, 2004 at 09:37 AM. Reason: Formatting
|
|
#3
September 24th, 2004, 09:41 AM
|
|||
|
|||
|
Many thanks
Very comprehensive thanks. I had missed the Linux forum, my host is actually running Linux not UNIX.
There's so much I have to learn about command line, and a lot of your post sort of makes sense to me from other operating systems. But I've been experimening with crontab by just typing in single commands. My host gives me SSH access and if I type something like: Code:
mysqldump --user=username --p=password --opt dbasename| gzip > /home/backup/dbsitebackup.gz it works fine, but if I try and run the same thing in crontab the .gz files is created empty. Why would that be? Is it more likely to be because I didn't 'bang-hash' or because I need the path for mysqldump, or I don't have permissions to run mysqldump in cron?
|
|
#4
September 24th, 2004, 10:04 AM
|
|||
|
|||
|
The principles should be the same in Linux. I don't think the korn shell comes as standard mind...
Code was: mysqldump --user=username --p=password --opt dbasename| gzip > /home/backup/dbsitebackup.gz The hash-bang just makes sure that a shell script is run in the one you designed it for even if the person running it is using a different shell themselves. It is irrelevant in this case. If you have permissions to run it from the command line you'll have permission to run it from cron. I reckon it'll be because cron can't find mysqldump so give it the full path. Of course, if you're going to put this command into a script you'll not need to give it the path to mysqldump, so long as you set the PATH variable up earlier in the script  Try the following modification too: mysqldump --user=username --p=password --opt dbasename| gzip > /home/backup/dbsitebackup.gz.`date +%a`
|
|
#5
September 25th, 2004, 01:32 AM
|
|||
|
|||
|
Redirects from the command line?
So is it possible to set up redirects on the server from the *NIX command line, or is that something that is done in Apache (about which I know less than nothing)?
|
|
#6
September 27th, 2004, 02:36 AM
|
|||
|
|||
|
I'm not sure what you mean by redirects...
*NIX is the ">" symbol, but I suspect you're talking more about than redirects of standard output. Can you please elaborate?
|
|
#7
September 27th, 2004, 10:34 AM
|
||||
|
||||
|
I think he means HTML redirects, where someone will access an index.html or whatever and it sends them to a different page.
I don't know of any way to do this manually, outside of some complicated file editing. Though, you could have 2 index files: 1 being the normal, the other being just a file with a redirect. Lets say, index.html and index.redir In cron rename the main file (index.html) to index.whatever, and index.redir to index.html. This'll send them where you want to go.
|
|
#8
September 28th, 2004, 11:15 AM
|
|||
|
|||
|
Thanks StevenC I used rename in the end.
bits working. Now I'm struggling with gzip to backup some other files. I cannot find any examples anywhere of the correct notation to backup a folder and all sub-folders. I've tried: gzip -r /path > zip.gz gzip -r /path/ > zip.gz gzip -r /path/* > zip.gz gzip -r /path/*.* > zip.gz The first 3 produce empty zip.gz files, the last one produces a syntax error. What am I doing wrong? (There are no files at /path, just other folders which themselves DO contain files.)
|
|
#9
September 28th, 2004, 12:15 PM
|
|||
|
|||
|
I think gzip works for singular files only. I've not used or tried to use it on multiple files. Most people will create a tar file of all the files first and then gzip that file.
Code:
#-- Tar up contents of data directory to data.tar, then gzip tar cvf data.tar data/* gzip data.tar #-- Unzip then untar gzip -d data.tar.gz tar xvf data.tar #-- Tar up and gzip in one step tar cvf - data/* |gzip -c > data.tar.gz #-- Ungzip and untar in one step gzip -dc data.tar.gz |tar xvf -
|
|
#10
September 28th, 2004, 12:57 PM
|
|||
|
|||
|
Thanks Andy, I had wondered wether this was the problem, but I'm none-plussed why if gzip is for single files, it has the -r switch.
http://www.gzip.org/#faq16 confirms your suspicions but the manpage for gzip says: Quote:
|
|
#11
September 28th, 2004, 01:11 PM
|
|||
|
|||
|
I've had a play.
It will get gzip to gzip files in subdirectories (see below). Code:
$ find . -type f -ls 60229 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file1 60231 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file2 60232 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file3 36188 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:07 ./file1 36189 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:07 ./file2 36185 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:07 ./file3 $ gzip * gzip: testdir is a directory -- ignored $ find . -type f -ls 60229 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file1 60231 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file2 60232 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file3 36188 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:07 ./file2.gz 36187 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:07 ./file1.gz 36189 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:07 ./file3.gz $ gzip -d * gzip: testdir is a directory -- ignored $ find . -type f -ls 60229 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file1 60231 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file2 60232 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:08 ./testdir/file3 36188 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:07 ./file3 36185 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:07 ./file1 36187 3 -r--r--r-- 1 oradba dba 2180 Sep 28 19:07 ./file2 $ gzip -r * $ find . -type f -ls 60231 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:08 ./testdir/file3.gz 60234 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:08 ./testdir/file1.gz 60229 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:08 ./testdir/file2.gz 36185 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:07 ./file2.gz 36189 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:07 ./file1.gz 36187 1 -r--r--r-- 1 oradba dba 876 Sep 28 19:07 ./file3.gz
|
|
| Viewing: Dev Shed Forums > Operating Systems > UNIX Help > Building a 'batch job' for crontab |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
 |
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
