#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    8
    Rep Power
    0

    Unable to iterate through csv reader


    I'm having a weird issue being able to read though a csv file. This is something I've done with out issue several times in the past. I even iterate through this csv file and write the contents to a temp file file with out issue but when I hit the following line of code it never enters the loop when it should.

    Code:
     if isMachineExpired(computer_last_connected) == True:
                    print "Machine Expired"
    
                    for item in file_schedule_reader:
                        print "in loop"
                        print item[0] + " : " + computer_guid
                        if item[0] == computer_guid: #Machine in schedule
    print "in loop" never happens but print "Machine Expired" does. I'm really at a loss as to why for item in file_schedule_reader: is not getting executed.

    Code:
    try:
        f = open(file_schedule,"rb")
        file_schedule_reader = csv.reader(f)
        f1 = open (file_csv_computer,"rb")
        f3 = open(file_schedule_temp,"wb")
        file_schedule_writer = csv.writer(f3)
        file_computer_reader = csv.reader(f1)
    
    
        #use the file_schedule_reader to build a temp scheudle file with the csv file schedule writer (can't interate with a writer object so we need to make a temp file)
        for item in file_schedule_reader:
            file_schedule_writer.writerow ([item[0],item[1],item[2],item[3],item[4]])
    
    
        for computer in file_computer_reader:
            if rownum != 0:
                computer_guid = computer[2]
                computer_last_connected = computer[9]
                computer_name = str(computer[1])
                computer_user_id = computer[7]
                computer_uniquename = Users[str(computer_user_id)]
                print "GUID: " + computer_guid + " connected: " + computer_last_connected + " name: " + computer_name + " user id: " + computer_user_id + " uniquename: " + computer_uniquename
                if isMachineExpired(computer_last_connected) == True:
                    print "Machine Expired"
    
                    for item in file_schedule_reader:
                        print "in loop"
                        print item[0] + " : " + computer_guid
                        if item[0] == computer_guid: #Machine in schedule
                            if item[4] == "True":
                                #do nothing this machine is an override
                                print "Machine is an override do nothing"
                                wow = true
                            else:
                                if isMachineExpired(item[3]) == True:
                                    print "Machine is old remove email the run team"
                                    #EMAIL RUN TEAM FOR REMOVAL
                        else: #Machine not in schedule add it
                            print "Machine not in schedule adding it now."
                            file_schedule_writer.writerow([computer_guid,computer_name,computer_uniquename,date_formated,"False"])
                            #email user
                            #build runteam email
            rownum +=1
    
    
    
    
    finally:
        f.flush()
        f3.flush()
        f.close() #close file schedule reader
        f1.close() #close the computer csv
        f2.close() #close the user csv
        f3.close() #close the file schedue writer
    This section of code however iterates just fine. (it's the same csv reader object reading the same file). Why can't I read through the csv file a second time? I even tried closing the file before reading it again and opening it again for reading but I still have the same problem.


    Code:
        #use the file_schedule_reader to build a temp scheudle file with the csv file schedule writer (can't interate with a writer object so we need to make a temp file)
        for item in file_schedule_reader:
            file_schedule_writer.writerow ([item[0],item[1],item[2],item[3],item[4]])
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2009
    Posts
    521
    Rep Power
    34
    There are no items in file_schedule_reader. Print the length before the for() loop. Once you read a file, the pointer is at the end of the file. If you try to read it again, it starts at the end, hence no records. You read the file once at the top, so you would have to initialize the pointer on every pass through "for computer in file_computer_reader:" You can close the file and then open it again, use seek() -on text files don't know about csv reader, or possibly better, read it as a text file using readlines() and iterate over the resulting list as many times as you want.
    Last edited by dwblas; February 8th, 2013 at 11:25 AM.
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    8
    Rep Power
    0
    Originally Posted by dwblas
    There are no items in file_schedule_reader. Print the length before the for() loop. Once you read a file, the pointer is at the end of the file. If you try to read it again, it starts at the end, hence no records. You can close the file and then open it again, use seek() -on text files don't know about csv reader, or possibly better, read it as a text file using readlines() and iterate over the resulting list as many times as you want.
    What's strange is the following code happens just before the section I'm having trouble with.

    Code:
        for item in file_schedule_reader:
            file_schedule_writer.writerow ([item[0],item[1],item[2],item[3],item[4]])
    This works just fine. It reads the first line of the csv; the header and writes it to a new csv file. There are actually no other records in the file.

    I also tried to close the file and reopen thinking I was at the end of the file but it still doesn't work.

    I would expect after closing and reopening the file I should at least enter the following section of code because the first entry in the CSV file is the header and it wouldn't match the computer GUID.

    Code:
    else: #Machine not in schedule add it
                            print "Machine not in schedule adding it now."
                            file_schedule_writer.writerow([computer_guid,computer_name,computer_uniquename,date_formated,"False"])
                            #email user
                            #build runteam email
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2009
    Posts
    521
    Rep Power
    34
    try using readlines and splitting on the comma. Note you might also have to strip() whitespace on some fields and maybe convert to integers if you compare number to number.
    Code:
    f = open(file_schedule,"rb").readlines()
    ...
    for item.split(",") in f:
    #
    # or
    for item in f:
        columns = item.split(",")
    ##
    ## split all items once and store in list if you do this many times
    You should also iterate over the input file once and store item[0] and [4] in a dictionary and then just look up file_schedule_reader instead of iterating through the whole file each time.
    Code:
                    for item in file_schedule_reader:
                        print "in loop"
                        print item[0] + " : " + computer_guid
                        if item[0] == computer_guid: #Machine in schedule
                            if item[4] == "True": 
    #
    # instead
                    if computer_guid in item_dict:
                        if item_dict[computer_guid] == "True"
    Last edited by dwblas; February 8th, 2013 at 11:38 AM.
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    8
    Rep Power
    0
    Originally Posted by dwblas
    try using readlines and splitting on the comma. Note you might also have to strip() whitespace on some fields and maybe convert to integers if you compare number to number.
    Code:
    f = open(file_schedule,"rb").readlines()
    ...
    for item.split(",") in f:
    #
    # or
    for item in f:
        columns = item.split(",")
    ##
    ## split all items once and store in list if you do this many times
    You should also iterate over the input file once and store item[0] and [4] in a dictionary and then just look up file_schedule_reader instead of iterating through the whole file each time.
    Code:
                    for item in file_schedule_reader:
                        print "in loop"
                        print item[0] + " : " + computer_guid
                        if item[0] == computer_guid: #Machine in schedule
                            if item[4] == "True": 
    #
    # instead
                    if computer_guid in item_dict:
                        if item_dict[computer_guid] == "True"
    How can I store 3 values to a key in a dictionary? Isn't a dictionary object a key:value pair? How can I store the other 3 additional pieces of information to the key?
  10. #6
  11. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    8
    Rep Power
    0
    Originally Posted by Murtasma
    How can I store 3 values to a key in a dictionary? Isn't a dictionary object a key:value pair? How can I store the other 3 additional pieces of information to the key?
    I think I got it working. While it's not pretty and I'm sure there's a better way to do it, it looks like it's working.
    Here is the new code
    Code:
    if isMachineExpired(computer_last_connected) == True:
                    print "Machine Expired"
                    f3 = open(file_schedule,"rb")
                    file_schedule_reader2 = csv.reader(f3)
    
                    for item in file_schedule_reader2:
                        print "in loop"
                        print item[0] + " : " + computer_guid
                        if item[0] == computer_guid: #Machine in schedule
                            if item[4] == "True":
                                #do nothing this machine is an override
                                print "Machine is an override do nothing"
                                wow = true
                            else:
                                if isMachineExpired(item[3]) == True:
                                    print "Machine is old remove email the run team"
                                    #EMAIL RUN TEAM FOR REMOVAL
                        else: #Machine not in schedule add it
                            print "Machine not in schedule adding it now."
                            file_schedule_writer.writerow([computer_guid,computer_name,computer_uniquename,date_formated,"False"])
                            #email user
                            #build runteam email
                    f3.close()
    Could I be running into issues trying to use an object that was defined outside of the if statement? I'm not terribly familiar with how things are compartmentalized in Python.
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    8
    Rep Power
    0
    had some duplicate of file open names. I fixed them up in my code. Caused some open file handles and things not getting properly written but I sorted that out. I'm sure there is a more elegant solution to this but it doesn't need to run fast and I can understand. I guess I'll leave it as is unless someone notices what was going on before or can explain it to me.

    Thanks for the suggestions .
  14. #8
  15. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2009
    Posts
    521
    Rep Power
    34
    How can I store 3 values to a key in a dictionary? Isn't a dictionary object a key:value pair? How can I store the other 3 additional pieces of information to the key?
    You can store as much as you like as the value, generally a list is used. If the records are not too long, you can store the entire record if it makes it easier for you to understand. Reading files is time consuming but if it runs fast enough as is, then it works OK as is.

IMN logo majestic logo threadwatch logo seochat tools logo