#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12

    Help IndexError: list index out of range


    Could somebody help me out and tell me what did I do wrong that I getting this error message "IndexError: list index out of range"

    Code:
    import os, re
    
    path = 'specify path'
    filelist = ['a','b','c','d']
    
    def lister(dummy, dirname, filesindir):
    	i = 0
    	count = 0
    	while i < len(filelist):
    		for fname in filesindir:
    			if os.path.isfile(os.path.join(dirname, fname)):
    				file = open(os.path.join(dirname, fname), 'r')
    				line = file.readline()
    				while line !="":
    					if re.search(filelist[i], line):
    						count += 1
    					line = file.readline() 
    				print filelist[i] + ' occured ' + str(count) + ' times'
    				i += 1
    
    os.path.walk(path, lister, None)
    Thanks
    Random
  2. #2
  3. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    What is the exact error message your getting including line numbers. I can however tell you that the error is referes to the fact that you are trying to access an element in a list or tuple that doen't exist. You might want to look at the newer os.walk() function.

    Try this something like this...

    Code:
    import os
    
    path = 'specify path'
    isin = ['a','b','c','d']
    
    def lister(dummy, pathname, filenames):
        count = 0
            for each in isin:
                for filename in filenames:
                    if os.path.isfile(os.path.join(pathname, filename)):
                        name = file(os.path.join(pathname, filename), 'r')
                        line = name.readline()
                        while line:
                            if isin[count] in line:
                                count = count + 1
                            line = name.readline()
                            print '%s occured %s times' % (isin[count], count)
    
    os.path.walk(path, lister, None)
    Note: untested code , just an example.

    Just out of interest what exactly do you want to do here, i'm sure there is a much nicer way to do this kind of thing using os.listdir(), just let us know what your trying to do .

    Have fun, hope this helps!

    Mark.
    Last edited by netytan; May 28th, 2004 at 09:57 AM.
    programming language development: www.netytan.com Hula

  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12
    Hi Mark,

    At work I am given a large directory of weblogs and a specific list and told to search all those logs and give the count of how many times they occur. So I decided to write this kind of script to do that plus learn Python. The original script which I have did not use the list and it was working without any errors. So I wanted to try and use the list to have those searching words passed right from the list instead of before I had a couple of "if re.search(string, line):" lines and it looked messy.

    The strange thing was that when I ran this script at work it would not generate an error when there were even number in the list and display the error when there were odd.

    The error I get is

    Traceback (most recent call last):
    File "D:\Temp Files\Python\filelist_v1.py", line 22, in ?
    os.path.walk(path, lister, None)
    File "C:\Python23\lib\ntpath.py", line 327, in walk
    func(arg, top, names)
    File "D:\Temp Files\Python\filelist_v1.py", line 16, in lister
    if re.search(filelist[i], line):
    IndexError: list index out of range

    I am sure there is a better way of writting this and if you can recomend something I would appreciate it. But it would need to be in a way that would be easy for me to modify it in the future.

    Thanks
    Random
  6. #4
  7. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    OK since you didnt mention walking a directory tree i'm going to assume that your log files are all in the one directory...

    Code:
    #!/usr/bin/env python
    
    import os
    
    def search(path, list):
        for name in os.listdir(path):
            full = os.path.join(path, name)
            if os.path.isfile(full):
                look = 0
                for line in file(full):
                    for each in list: print '%s occured %s times' % (look, line.count(each))
    
    search('/', ('a', 'b'))
    Note: again this is untested but it should work. Give it a go and let me know.

    Mark.
    programming language development: www.netytan.com Hula

  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12
    Hi Mark,

    Yes it is walking a tree. I did not mentioned it because I thought thats what I have done in my original code. I ran your code unchanged and this is what I got

    0 occured 0 times
    0 occured 0 times
    0 occured 0 times
    0 occured 3 times
    0 occured 0 times
    etc

    Is this right?

    Thanks
    Random
  10. #6
  11. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Oops, gibe this a go. It doesn't actually walk a dir-tree but it will preform this action for each each file in a directory - or thats the idea.

    Code:
    #!/usr/bin/env python
    
    import os
    
    def search(path, list):
        for name in os.listdir(path):
            full = os.path.join(path, name)
            if os.path.isfile(full):
                test = file(full, 'r').read()
                for each in list: print '%s occured %s times in %s' % (each, test.count(each), full)
    
    search('/', ('a', 'b'))
    You could of course call this function recursivly from within itself to walk the tree.

    Mark.
    Last edited by netytan; May 29th, 2004 at 06:11 AM.
    programming language development: www.netytan.com Hula

  12. #7
  13. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    If this doesn't work for you then check out os.walk(), assuming you really want to walk over a directory tree. It's a little hard to help on this one since i dont have Python right now so can't test anything - which makes it hard to help much

    http://www.python.org/doc/2.3.3/lib/os-file-dir.html

    Mark.
    Last edited by netytan; May 29th, 2004 at 06:02 AM.
    programming language development: www.netytan.com Hula

  14. #8
  15. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Just been thinking and if all you need to do is to search a bunch of files and count how many times a string occures inside them - and you know the location of these files - then you can just pass a sequence of paths to a function.

    Code:
    #!/usr/bin/env python
    
    def search(paths, count):
        for path in paths:
            test = file(path, 'r').read()
            for each in count:
                print '%s occured %s times in %s' % (each, test.count(each), path)
    
    search(('group', '/of/log', 'files.txt'), ('look', 'for', 'these'))
    Mark.
    programming language development: www.netytan.com Hula

  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12
    Hi Mark,

    Thanks for the code, I will try it out and let you know how it goes. But my only question is why my code gave the error it did? Any ideas?
    Cheers.
    Random
  18. #10
  19. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Sorry i thought we covered this one . The reason your code threw an IndexError was because you were trying to access an element in filelist that didn't exist. Try printing i after incrementing it and you should be able to see where it's going wrong .

    Good luck,

    Mark.
    programming language development: www.netytan.com Hula

  20. #11
  21. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12
    I know that the error comes from the element that does not exist but my qiestion is why it enters the while loop again?
    Is there a way to fix that?

    Thanks
    Random
  22. #12
  23. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    The problem comes form the fact that your incrementing i inside the for loop (printing i as recomended should show this). Unindent that so that it happens after the for loop has finished and the IndexError should go away .

    Mark
    programming language development: www.netytan.com Hula

  24. #13
  25. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12
    Thanks Mark that worked.

IMN logo majestic logo threadwatch logo seochat tools logo