#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2004
    Posts
    9
    Rep Power
    0

    Question Is Python up to the task?


    Im going to be starting out a new project and i just wanna know if python will beable to handle my needs.

    1. Beable to parse a line of text for keywords.
    2. Enter the keywords if found into a database (prob. mySQL)
    3. do this in upwards of 100lines per seconds.

    Ofcourse the project will be more complex than this but thats the basics. The input source wont be static like a text file or anything thats why its important that it can handle if theres a sudden burst.
    I wont be need a gui or anything, it will be ran on linux so all im going to do is have a text output when ever i type the command to get one.
    Last edited by wrecher; November 19th, 2004 at 10:56 AM. Reason: spelling
  2. #2
  3. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2003
    Location
    London, ON, Canada, eh
    Posts
    127
    Rep Power
    12
    Not sure how to do this with python but it would be easy in tcl/tk.

    1. foreach keyword $keywords {
    set kw [lsearch $line_of_text $keyword]
    }
    2. catch {
    eval exec <external linux command> $keyword
    }
    3. Of course this'll depend on your system but I believe one
    tcl/tk call is equivalent to 3 C calls.

    I managed to read 661 lines from a text in under a second.

    Originally Posted by wrecher
    Im going to be starting out a new project and i just wanna know if python will beable to handle my needs.

    1. Beable to parse a line of text for keywords.
    2. Enter the keywords if found into a database (prob. mySQL)
    3. do this in upwards of 100lines per seconds.

    Ofcourse the project will be more complex than this but thats the basics. The input source wont be static like a text file or anything thats why its important that it can handle if theres a sudden burst.
    I wont be need a gui or anything, it will be ran on linux so all im going to do is have a text output when ever i type the command to get one.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2003
    Posts
    624
    Rep Power
    35
    1. Beable to parse a line of text for keywords.
    What does "parsing a line for keywords" consist of?

    3. do this in upwards of 100lines per seconds.
    How long is a line?
    How fast is the computer?
    What else is it doing besides running this script?
    Is the DB on the same machine or accessed over a network?
    Do you need things like transactions for each DB insert?

    Basically, I think you should try it.
    (not real working code).
    Code:
    import stuff
    
    db = connect_to_db('details, username, password')
    cursor = db.getCursor()
    fin = open("test_file", "rb")
    
    for line in fin:
        result = test_search_for_keywords(line)
        if result:
            cursor.execute("insert into table values (?,?)", result)
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Feb 2004
    Location
    London, England
    Posts
    1,585
    Rep Power
    1373
    I am sure Python is up to the job, although as sfb pointed out there are a lot of unknowns.

    The bottleneck in a task like this is almost certainly going to be the database access, so the speed of the language will be largely irrelevant.

    Dave - The Developers' Coach

IMN logo majestic logo threadwatch logo seochat tools logo