#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11

    Search a file for text


    Hello!

    I have just started to learn python and wanted to write my first useful script. At work I do a search thru very large web log files.
    So I wanted to create a script that would work like M$ search
    text in files, where I would list a path and have all files where
    the text occurs be written in separate log file. So far my search
    in Google has not produced any sample code that would show
    how to acomplish this.
    Still learning and trying
    tia
  2. #2
  3. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Hi random,

    This shouldn't be very hard. I'm assuming that the log's are all in the same dir? if so then it's even easier!

    Code:
    #!/usr/bin/env python
    
    path = 'directory to search'
    string = 'search string'
    
    found = []
    
    #open path and read a list of files
    for file in os.listdir(path):
        #if 'file' is a file
        if os.path.isfile(path + file):
            #see if string is in file
            if string in open(path + file, 'r').read():
              #add file to found list.
                found.append(file)
    
    #print files that contain the search text.            
    for file in found:
        print file
    This is just the basic idea, I havn't tested this but it should work. let me knwo if it doesn't

    Hope this some help,
    Mark.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11
    Hi Mark,

    Thanks for the speedy reply. As I am just starting to learn python
    the comments that you have provided are going to be extremly
    helpful for me to understand and learn. I have a couple of web logs at home which I would try this on. Will let you know?

    Again thanks alot.
    Random
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11
    Hi Mark!

    I am getting an error by running this script, and not quite sure how to fix it?

    tia
    random

    File "D:\Python\search.py", line 13, in ?
    if string in open(path + file, 'r').read():
    TypeError: 'in <string>' requires character as left operand
  8. #5
  9. onCsdfeu
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Location
    Canada
    Posts
    100
    Rep Power
    11
    Just in case : have you changed the values of 'directory to search' and 'search string' ? I know it happens to me sometimes.

    And if you did, what were those directory and search string ? Maybe you put some illegal character or whatnot. Just put 'em here.
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11
    All I did was to add

    import os
    path = 'C:/'
    string = 'foo'

    Was there anything that I forgot to change or supply?
  12. #7
  13. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    I recreated the program above and it works fine for me (I've attached the source code). I've never seen that error before but if you post your code i'll take a look.. What version of Python are you using anyway?

    It may be worth saying that path's in windows use '\' instead of '/' (you'll have to escape backslashes ('\\') when you use them in strings)

    Have fun,
    Mark.
    Attached Files
  14. #8
  15. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11
    Thanks for trying to help me get this code running, but for me it does not work. I took your code and copied it exactly like you did
    and ran it. My version of python is 2.2.

    Error message

    C:\>search.py
    Traceback (most recent call last):
    File "C:\search.py", line 12, in ?
    if string in open(path + file, 'rb').read():
    TypeError: 'in <string>' requires character as left operand
  16. #9
  17. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    I'm running Python 2.3, maybe this is something to do with that.. you could try renaming the string variable (although string isn't a reserved word).

    Alternativle you could try using count instead of..

    if open(path + file, 'rb').read().count(string) != 0: found.append(file)

    Mark.
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11
    Upgrading to 2.3 made the error go away.
    Thanks
  20. #11
  21. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Very welcome, glad I could help. 2.3 is very nice hope you like it, it's faster than 2.2, which seems to be the main improvment. That and a few nice module additions and improvments.

    http://forums.devshed.com/t73300/s.html

    Mark.

IMN logo majestic logo threadwatch logo seochat tools logo