#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2012
    Posts
    8
    Rep Power
    0

    Multiprocessing and performance issues


    I have the following code to brute-force a numeric password below 80000.

    Code:
    import multiprocessing
    import hashlib
    import time
    oldTime = time.time()
    def cracker(start,end,checkStr):
        curStr = start
        while curStr != end:
            digest = hashlib.sha1()
            digest.update(str(curStr))
            if checkStr == digest.hexdigest():
                print "Match found at " + str(curStr)
                newTime = time.time()
                print (newTime - oldTime)
            curStr += 1
    if __name__ == '__main__':
        procs = []
        j = 0
        for i in range(8):
            p = multiprocessing.Process(target = cracker,args = (j,j*10000,"a0849622401ebc624406648bc39eca144427cd90"))
            j = j*10000 + 1
            procs.append(p)
            p.start()
        for p in procs:
            p.join()
    A syntax highlighted version

    The problem here is that it still takes 0.475 seconds for the execution.I tested the speed of my processor in checking against hashes as 1.36 Lac hashes per second.So this plain-text "60000" should be cracked in almost no time.But it still takes 0.475 seconds i.e almost 60000+ hashes. How do I get it to work as expected i.e it should be cracked in almost 0.07 seconds ?
  2. #2
  3. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,837
    Rep Power
    480
    Changing cracker to print the time it takes to start may help you understand the performance.
    Code:
    def cracker(start,end,checkStr):
        curStr = start
        print ('time to start: {}'.format(time.time() - oldTime))
        while curStr != end:
            digest = hashlib.sha1()
            digest.update(str(curStr))
            if checkStr == digest.hexdigest():
                print "Match found at " + str(curStr)
                newTime = time.time()
                print (newTime - oldTime)
            curStr += 1
    Ah! Better yet, see if
    time.process_time()
    helps. New in python3.3

    timeit module or python profiler might help?

    The point is you need to count process time, not wall clock time.
    [code]Code tags[/code] are essential for python code and Makefiles!
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2012
    Posts
    8
    Rep Power
    0
    Thanks for the info.But running the exact same code on Python 3.3.0 produces no output.I haven't changed the file a bit.Also tried to modify the syntax of print but no effect.

    Could you please tell what happened here ?


    EDIT:Also updated the digest.update() by writing:

    Code:
    digest.update(str(curStr).encode('utf-8'))
    But it still prints not-a-thing
  6. #4
  7. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,837
    Rep Power
    480
    There you go! You're on the way. Process time is 1/20th of a second to first crack like you suggested.

    print is a python3 function.
    Invalid: print 'string' # syntax error
    Correct: print('string') # as function call
    Code:
    import multiprocessing
    import hashlib
    import time
    oldTime = time.process_time()
    
    def cracker(start,end,checkStr):
        curStr = start
        while curStr != end:
            digest = hashlib.sha1()
            digest.update(str(curStr).encode('utf-8'))
            if checkStr == digest.hexdigest():
                print( "Match found at " + str(curStr))
                newTime = time.process_time()
                print (newTime - oldTime)
            curStr += 1
    if __name__ == '__main__':
        procs = []
        j = 0
        for i in range(8):
            p = multiprocessing.Process(target = cracker,args = (j,j*10000,"a0849622401ebc624406648bc39eca144427cd90"))
            j = j*10000 + 1
            procs.append(p)
            p.start()
        for p in procs:
            p.join()
    [code]Code tags[/code] are essential for python code and Makefiles!
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2012
    Posts
    8
    Rep Power
    0

    Thumbs up Still output-less


    My code was just like the one you posted.But it prints nothing and like the loop is going on for ever.The script doesn't end on the IDLE.So I tried by running the script from the command line and yes it gives the output(fine here) but the time is still around 0.655s



    Any ideas henceforth ?

    Thanks for the help till here.
    Last edited by anuvab1911; March 20th, 2013 at 03:26 PM. Reason: Typo
  10. #6
  11. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,837
    Rep Power
    480

    Agreed


    Interesting, when I run that program in idle for 3.3 it doesn't do anything except hang. I watched my cpu monitor. Uses no discernable cpu time.
    Code:
    import multiprocessing
    import hashlib
    import time
    
    oldTime = time.process_time()
    
    def cracker(start,end,checkStr):
        curStr = start
        while curStr != end:
            digest = hashlib.sha1()
            digest.update(str(curStr).encode('utf-8'))
            if checkStr == digest.hexdigest():
                print( "Match found at " + str(curStr))
                newTime = time.process_time()
                print (newTime - oldTime)
            curStr += 1
    
    def main():
        procs = []
        j = 0
        for i in range(8):
            p = multiprocessing.Process(target = cracker,args = (j,j*10000,"a0849622401ebc624406648bc39eca144427cd90"))
            j = j*10000 + 1
            procs.append(p)
            p.start()
        for p in procs:
            p.join()
    
    main()
    [code]Code tags[/code] are essential for python code and Makefiles!
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2009
    Posts
    474
    Rep Power
    33
    Are you using a multi-core machine? If not, 8 processes is possibly too many for one core to do efficiently. Start with 2 processes and increment up to see what works best. Also remove the print statements where possible as I/O is the slowest part of today's computer. But all of this appears to be a very small improvement.
    Code:
        output = []
        while curStr != end:    
            digest = hashlib.sha1()
            digest.update(str(curStr).encode('utf-8'))
            if checkStr == digest.hexdigest():
    ##            print "Match found at " + str(curStr)
                newTime = time.time()
    ##            print (newTime - oldTime)
                output.append([cur_Str, newTime-oldTime])
            ## other stuff
        print output
    Last edited by dwblas; March 20th, 2013 at 05:07 PM.
  14. #8
  15. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2012
    Posts
    8
    Rep Power
    0
    Yes, I am on an i7 with 8 logical cores. That's why I am trying to spawn 8 processes. Another interesting thing I noticed was that after running the program,since the IDLE apparently freezes,so I close that forcibly using the Task manager. But in the.processes tab,there are another 7 pythow.exe instances running. Memory usage went upto 78% during that time.
  16. #9
  17. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2012
    Posts
    8
    Rep Power
    0

    Check this


    @b49P23TIvg

    Check this

IMN logo majestic logo threadwatch logo seochat tools logo