Page 2 - [BASH] Issue with parsing line from file

Dev Shed Forums Sponsor:

#16

September 30th, 2011, 11:16 AM

akaballa123

Registered User

Join Date: Sep 2011

Posts: 23

Time spent in forums: 5 h 5 m 10 sec
Reputation Power: 0

Quote:

Originally Posted by SimonJM

Just show the plain output of a cat of your script

this is the plain output of my script, if this is what your asking for.

Code:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++
Enter the input file name ( must be a txt file).
If the Input file is not in the same directory as this script then please enter the full dicrectory name of the input file.
NOTE: please enter "null" if start or skip fields are empty: 
input.txt
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++
Specify number of reports to generate per hour : 1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++
/app/mxjava/gpm_pat07/tds/
process: CreateTradeDataFeed
timestamp: 20110929
report type: before
skip: GPM_FIXINGS,GPM_RAW_CASH
start: null
runnodes: null
Please wait patiently while Reports are being generated...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++
Current Process: CreateTradeDataFeed
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++
./createlist.sh: line 112: [: too many arguments
./createlist.sh: line 129: [: too many arguments
./createlist.sh: line 147: [: too many arguments
./createlist.sh: line 152: [: too many arguments
./createlist.sh: line 159: [: too many arguments
./createlist.sh: line 165: [: too many arguments
<FeedName>${TDS_HOME_DIR}/output/CreateTradeDataFeed/mucm_trade-data_curr-fut-fut_${LAST_BUS_DAY}.csv</FeedName>
report file name: 
 
20110929
output directory: /app/mxjava/gpm_pat07/tds/output/CreateTradeDataFeed/
chmod: WARNING: can't access /app/mxjava/gpm_pat07/tds/output/CreateTradeDataFeed/mucm_trade-data_curr-fut-fut_20110929.csv
mv: cannot access /app/mxjava/gpm_pat07/tds/output/CreateTradeDataFeed/mucm_trade-data_curr-fut-fut_20110929.csv

this is my code:

Code:

#!/usr/bin/bash
#set -x

jBPMarray=("$@")
num_processes=${#jBPMarray[@]} 
output_dir=""
rpt_file_name=""
time_itr=0
index_itr=0
itr=0
date_itr=1
time_itr=0


echo -n "Specify number of reports to generate per hour : "
read num_reports_per_hr

#TODO: error checkinf for user unput of number of reports to be generated per hour
#if [ $num_reports_per_hr > $num_processes ]
#then
#      echo "Number of reports generated per hour cannot be greater than total number of reports to be processed "
#      exit 0
#fi


#TODO: check validity of timestamp
echo -n "Specify a timestamp, e.g. ( yyyymmdd ) : "
read time_stamp

#prompt user if this report is the before or after release
echo -n "Is this report the before or after copy.  Please Enter before|after: "
read report_type

#get user input for timestamp of each jBPM process
#for jBPM in "${jBPMarray[@]}"
#do
 #   echo -n "Specify a timestamp for $jBPM, e.g. ( yyyymmdd ) : "
  #  read time_stamp[$time_itr];
   # echo "timestamp: "
  #  echo ${time_stamp[$time_itr]}
   # time_itr=$(($time_itr + 1))
#done

#Set current home directory
#CUR_EQD_DIR=` $0 | sed  's/tds.*$//g'`
CUR_EQD_DIR=`pwd | sed  's/tds.*$//g'`
CUR_EQD_DIR="${CUR_EQD_DIR}tds/"
echo $CUR_EQD_DIR

for jBPMprocess in "${jBPMarray[@]}"
do  
    
    #store date to append to filename
    datestamp=`date +%Y%m%d` 
    
    #define list of report file structure
    #Do not think this is required!!
    #if [ "$date_itr" -eq 1 ]
#    then
 #         echo "[Report File Name - Output Directory - Timestamp]" >> list_report_${datestamp}.txt
  #  fi
    date_itr=$(($date_itr + 1))
    
    
    echo "Please wait patiently while Reports are being generated..."
    
    echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++"
    echo "Current Process: $jBPMprocess"
    echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  ++++++++++"
    
    
    #prompt user if any nodes need to be skipped
    echo -n "Would you like to skip any nodes? Please specify the node <node1>:  "
    read skip_node
    echo $skip_node
    #prompt user if any nodes need to be started
    echo -n "Would you like to start any nodes? Please specify the nodes <node1>:  "
    read start_node
    echo $skip_node
    NEW_DIR="scripts/reports/EQDReports.sh"
    FULL_DIR="$CUR_EQD_DIR$NEW_DIR"
    
    #TODO: Work on adding start stop
    if [[ $skip_node -eq "" && $start_node -eq "" ]]
    then
          "$FULL_DIR" "$jBPMprocess" "-vs LAST_BUS_DAY=$time_stamp"
    fi
    if [[ $skip_node -ne "" && $start_node -eq "" ]]
    then
          "$FULL_DIR" "$jBPMprocess" "-vs LAST_BUS_DAY=$time_stamp" "-skip $skip_node"
    fi
    if [[ $skip_node -eq "" && $start_node -ne "" ]]
    then
          "$FULL_DIR" "$jBPMprocess" "-vs LAST_BUS_DAY=$time_stamp" "-skip $start_node"
    fi
    if [[ $skip_node -ne "" && $start_node -ne "" ]]
    then
          "$FULL_DIR" "$jBPMprocess" "-vs LAST_BUS_DAY=$time_stamp" "-skip $skip_node" "-start $start_node"
    fi
    itr=$(($itr + 1))
    mod=$(($itr % $num_reports_per_hr))
    
    #grep for the first occurrence of output directory/file in processdefinition file
    output_file=`grep .*output.*\$\{LAST_BUS_DAY\}.csv jBPM/"$jBPMprocess"/processdefinition.xml | head -1`
    echo $output_file
    time_style="\$\{LAST_BUS_DAY\}"
    
    #TODO: There are more timestamp identifiers (CURRENT_BUS_DAY, ...) fix these! 
    if [ $output_file -eq ""]
    then
        output_file=`grep .*output.*\$\{CURRENT_TIMESTAMP\}.csv   jBPM/"$jBPMprocess"/processdefinition.xml | head -1`
        time_style="\$\{CURRENT_TIMESTAMP\}"
        
        if [ $output_file -eq ""]
        then
            
            output_file=`grep .*output.*\$\{CURRENT_BUS_DAY\}.csv   jBPM/"$jBPMprocess"/processdefinition.xml | head -1`
            time_style="\$\{CURRENT_BUS_DAY\}"
            
            if [ $output_file -eq ""]
            then
            
                output_file=`grep .*output.*\$\{LAST_11_BUS_DAY\}.csv   jBPM/"$jBPMprocess"/processdefinition.xml | head -1`
                time_style="\$\{LAST_11_BUS_DAY\}"
                
                if [ $output_file -eq ""]
                then
            
                    output_file=`grep .*output.*\$\{LAST_2_BUS_DAY\}.csv   jBPM/"$jBPMprocess"/processdefinition.xml | head -1`
                    time_style="\$\{LAST_2_BUS_DAY\}"
                fi
            fi
        fi
     fi        
        # read grep result into multiple variables
        read ADDR1 ADDR2 ADDR3 ADDR4 <<<$(IFS="\/"; echo $output_file)
    
        #remove opening feedname tag from first addr
        ADDR1=${ADDR1#<*>}; 
        echo " first addr: "
        echo $ADDR1
        echo "report file name: "
        echo $rpt_file
    
        #extract report file name and remove closing FeedName tag
        time_stamp=${time_stamp#_}
        echo $time_stamp
        rpt_file="$ADDR4"  # 
        rpt_file=${rpt_file%<*>} 
        rpt_file=${rpt_file/${time_style}/${time_stamp}}
        rpt_file_init=$rpt_file

   
    #append type of release to report file name and
    rpt_file="${rpt_file}"_"$report_type"
    
    #Concatenate output dir
    output_dir="$ADDR1/$ADDR2/$ADDR3/"
    output_dir=${output_dir/\$\{TDS_HOME_DIR\}\//$CUR_EQD_DIR}
    echo output directory: $output_dir
    echo "$rpt_file - $output_dir - $time_stamp" >> list_report_${datestamp}.txt
    
    #TODO: mv command is giving trouble with access
    #change the rpt file name in directory to accomodate before / after
    CHMOD=`chmod 777 $output_dir$rpt_file_init` 
    CH_FILE_TYPE=`mv ${output_dir}${rpt_file_init} ${output_dir}${rpt_file}`
    
    if [ "$mod" -eq 0 ]
    then 
        sleep 20 
    fi
    
   # time_itr=$(($time_itr + 1))
    
done

#17

September 30th, 2011, 10:32 PM

SimonJM

Contributing User

Join Date: Mar 2006

Posts: 2,108

Time spent in forums: 1 Month 1 Week 1 Day 4 h 50 m 50 sec
Reputation Power: 1485

When testing for empty strings (or in fact any strings) I find it best (if not critical) to enclose BOTH sides of the test in quotes, so
[[ $a == "" ]] would become [[ "$a" == "" ]]

__________________
The moon on the one hand, the dawn on the other:
The moon is my sister, the dawn is my brother.
The moon on my left and the dawn on my right.
My brother, good morning: my sister, good night.
-- Hilaire Belloc

#18

October 4th, 2011, 10:32 AM

akaballa123

Registered User

Join Date: Sep 2011

Posts: 23

Time spent in forums: 5 h 5 m 10 sec
Reputation Power: 0

My apologies,

I did not see this reply earlier. I followed what you did and it works just fine! Thank you!

I also have a very small question. I am trying to run a perl script from inside my shell script. This is the code im running:

Code:


##provide inputs and run comparison tools
    "perl ${OUTPUT_DIR}comparereport.pl --keys=$key_str --ignore=$igr_str ${OUTPUT_DIR}$RPT_before ${OUTPUT_DIR}$RPT_after $TIME >> ${OUTPUT_DIR}$RPT"

where ${OUTPUT_DIR} contains the full directory to the files I am inputting as arguments to the perl script. Basically I get a "no such file or directory" error:

Code:


perl /app/mxjava/gpm_pat07/tds/output/CreateVegaFeed/comparereport.pl --keys=M_DEALNUM,VOL_SKEW_UP, --ignore=M_PTFOLIO, /app/mxjava/gpm_pat07/tds/output/CreateVegaFeed/mucm_vega_report_20111003.csv_before /app/mxjava/gpm_pat07/tds/output/CreateVegaFeed/mucm_vega_report_20111003.csv_after 20111003 >> /app/mxjava/gpm_pat07/tds/output/CreateVegaFeed/mucm_vega_report_20111003_comparison_output.csv: No such file or directory

I then ran this code directly from the terminal and it works fine. Am not sure If I am inputting correctly in the shell script. I also tried to run it with `` but with no avail. Please advice. Thanks!

#19

October 4th, 2011, 03:37 PM

SimonJM

Contributing User

Join Date: Mar 2006

Posts: 2,108

Time spent in forums: 1 Month 1 Week 1 Day 4 h 50 m 50 sec
Reputation Power: 1485

Is this script being run from cron? If so your environment, especially the PATH variable will be very basic. Quick way around that is to, from the command line, run a which perl command, note the full path and use that in your script.
If it's not being run from cron then ... I'd have to suspect that the content of variable OUTPUT_DIR does not terminate in a /
A final thing, it is not usual to put a command you are executing in quotes.
One possible thing to try is to put an echo in front of the command, run the script by hand then copy/paste the output from the echo onto the command line to make sure it works. If not you'll need to carefully look over the command line that is trying to be run to spot any issues.

Page 2 - [BASH] Issue with parsing line from file

Developer Shed Advertisers and Affiliates