#16
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2004
    Posts
    3
    Rep Power
    0

    Lightbulb Performance tweaks


    Hi,

    just a note about performance: if you have the choice between code readability and performance, it's usually better to chose readability, unless you are really certain you are inside a hotspot of your code. If you want to check if your code tuning really did change execution time, use the timer module. If you post performance tips here, a sample timer output can be helpful.

    Code:
    import timing
    
    def f():
        timing.start()
        x=[]
        for i in xrange(1,10000):
            x.append(str(i))
        s = ''.join(x)
        timing.finish()
        return timing.milli()
    
    def f2():
        timing.start()
        x=''
        for i in xrange(1, 10000):
            x += str(i)
        timing.finish()
        return timing.milli()
    
    print "Creating string with append: %sms" % f2()
    print "Creating string with list.join: %sms" % f()
    which looks like this (don't laugh about my old cpu: ):

    Creating string with append: 225ms
    Creating string with list.join: 58ms
  2. #17
  3. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Maybe this is a Windows thing but there doesn't appear to be a 'timer' or 'timing' module in the standard library. If this is a third party module maby you could post a link to it's homepage .

    So for those of you out there who are scratching there heads and saying "ok, now we know why that didnt work... but how do we check preformance ", 'timeit'.

    http://www.python.org/doc/2.3.3/lib/module-timeit.html

    Pythons timeit module (new in Python 2.3 - consider using Pystone or one of the various preformance tracking modules to check preformance with earlier Python versions) makes checking code snippets surprisingly easy!

    Code:
    >>> import timeit
    >>> 
    >>> def one(string): return string
    ...     
    >>> tone = timeit.Timer("one('string')", "from __main__ import one")
    >>> tone.timeit()
    1.2665732655966053
    >>> timeit.Timer('None').timeit()
    0.36021980447235608
    >>> timeit.Timer('pass').timeit()
    0.15026267304922847
    >>>
    Edit: Links to other main Python profile modules.

    http://www.python.org/doc/2.3.3/lib/module-hotshot.html
    http://www.python.org/doc/2.3.3/lib/module-profile.html

    Mark.
    Last edited by netytan; February 25th, 2004 at 05:01 PM.
    programming language development: www.netytan.com – Hula

  4. #18
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2004
    Posts
    3
    Rep Power
    0

    Cool Oops


    Originally Posted by netytan
    Maybe this is a Windows thing but there doesn't appear to be a 'timer' or 'timing' module in the standard library.
    Yes, I'm sorry, I didn't expect to find modules in my installation that weren't part of the standard lib. Somehow my distribution installed a C-Style library with python bindings called timing, perhaps it's part of the C-profiler for python under linux, don't know. Glad you catched this before I cluttered too many files with it, would have been portability nightmare .

    Next time I find something using help() I'll check with the docs if it really is a standard module. That's a nice tip, too .
  6. #19
  7. Mini me.
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2003
    Location
    Cambridge, UK
    Posts
    783
    Rep Power
    13

    Posting Code


    If you want help - it is a lot easier for people to read your code if you use the # button on the submit form to make a code block that preserves the layout.
    Last edited by Grim Archon; March 17th, 2004 at 03:32 AM.
  8. #20
  9. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Another nice piece of functionality (related to Grims post) is that you can highlight a section of text in you're post and hit pretty much any of the buttons and the respective tags are placed around the highlighted area!

    Mark.
    programming language development: www.netytan.com – Hula

  10. #21
  11. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2004
    Posts
    1
    Rep Power
    0

    A little tip


    Tip: During the active shell console in python, you can ask python for documentation as follows:

    Code:
    # This is a fake function
    def myFakeFcn():
        """ These are the commented
                documents of my function named
            myFakeFcn()         where spacing is easily
        preserved  !"""
    
        print "This function has no purpose!!",
    
    print myFakeFcn.__doc__  #this will output the documentation
    >>> 
     These are the commented
                documents of my function named
            myFakeFcn()         where spacing is easily
        preserved  !
    I know another post stated that the __doc__ could be used as a reference, but I wanted to point out that it can be used during an active shell.
  12. #22
  13. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    This is a commonly used method of adding test code directly into your modules or even telling your program what to do when its run. In any case very usfull stuff... "But we already know this?!"... Most do but as it seems to come up periodically so i thought its worth mentioning here .

    Code:
    #!/usr/bin/env python
    
    #Any code that you want to be run when Python interprets
    #the file. Imported or run.
    
    if __name__ == '__main__':
    
        #This part of the program will only be run if the program
        #if executed (is running). If the program is imported then
        #Python will ignore this part of the program =].
    Note: this isnt a must i.e. main in C/C++. Its just a nice way to organise your programs.

    Take care guys.

    Mark.
    programming language development: www.netytan.com – Hula

  14. #23
  15. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Feb 2003
    Location
    Canada
    Posts
    543
    Rep Power
    24
    Link: http://www.onlamp.com/python/pythonc...tion.csp?day=1

    Link in detail: Recipe of the day. Every day, a recipe from the Python Cookbook is featured.

    I just purchased Python Cookbook (edited by Alex Martelli & David Ascher). By the way, I highly recommend this book.

    Here is what's on the back of the book:

    "The Python Cookbook is a collection of problems, solutions, and practical examples for Python programmers, written by Python programmers. Over the past year, members of the Python community have contributed material to an online repository of Python recipes hosted by ActiveState. This book contains the best of those recipes, accompanied by overviews and background material by key Python figures.

    The recipes in the Python Cookbook range from simple tasks, such as working with dictionaries and list comprehensions, to entire modules that demonstrate complex tasks, such as a templating system and network monitoring. This book contains over 200 recipes.

    This book is a treasure trove of useful code for all Python programmers, from novices to advanced practioners, with contributions from such Python luminaries as Guido van Rossum, David Ascher, Tim Peters, Paul Prescod, Mark Hammond, and Alex Martelli, as well as over 100 other Python programmers. The recipes highlight Python best practices and can be used directly in day-to-day programming tasks, as a source of ideas, or as a way to learn more about Python.

    The recipes in the Python Cookbook were edited by David Ascher, who is on the board of the Python Software Foundation and is the co-author of Learning Python, and Alex Martelli, who is known for his numerous and exhaustive postings on the Python mailing list. The book contains a foreword by Guido van Rossum, the creator of Python."

    Read this book on-line.
  16. #24
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    11
    I thought that this would benefit those who are like me learning python and need a little help in understanding what is going on with their code. You can walk step by step and see what is happening line by line, this is similar to debugging in VB. Hope someone finds it useful in their learning proccess.

    Download from
    https://sourceforge.net/projects/hapdebugger/
  18. #25
  19. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2004
    Posts
    27
    Rep Power
    0

    Creating a Local __dict__


    Accessing the local namespace dict is already done by calling the builtin method locals(). However, the limitation of doing this is that modifying the dict returned by locals() is not considered 'safe'. Safe in the sense that any modifications are not guaranteed to be reflected in the actual namespace.

    Another approach I have taken is to access the local dict via:
    Code:
    sys._getframe().f_locals
    From all of the documentation I've read and input I've received, the f_locals seems 'safe' for modification.

    Using this approach, I have created a psuedo local __dict__. Here is a snippet of the main script code to do this:
    Code:
    import sys
    
    __all__ = ['local_dict']
    
    bad = ['__init__', '__new__', '__repr__']
    
    
    ############################################################
    class __dict( dict ):
        """
        Wrapper to mimic a local dict.
    
        Written by Derrick Wallace
        """
    
        def __init__( self, *args ):
            dict.__init__( self, *args )
    
            for attr in dict.__dict__:
                if callable( dict.__dict__[attr] ) and ( not attr in bad ):
                    exec( 'def %s(self, *args): return dict.%s(sys._getframe(1).f_locals, *args)'%( attr, attr ) )
                    exec( '__dict.%s = %s'%( attr, attr ) )
    
        #####################################
        def __repr__( self, *args ):
            # Must implement repr to prevent recursion
            if sys._getframe(1).f_code == sys._getframe().f_code:
                return '{...}'
            return dict.__repr__( sys._getframe(1).f_locals, *args )
    
    
    local_dict = __dict()
    I've only 'wrapped' some basic dict methods here, but all of them should and can be wrapped. Notice the repr method is different in that it must perform a check, this is to prevent recursion. In order to finish implementing this as a psuedo local __dict__ in another script, it must be imported. Assuming the above script is saved in a file called local_test.py, then this is how it could be done:
    Code:
    from local_test import local_dict as __dict__
    Now __dict__ acts like the local __dict__ for ANY namespace, whether it be a class method or a function. I just think that is cool! For example:
    Code:
    >>> from local_test import ldict as __dict__
    >>> __dict__
    {'__builtins__': <module '__builtin__' (built-in)>, '__file__': 'C:\\Python23\\Scripts\\pyshell', '__dict__': N/A, '__name__': '__main__', '__doc__': None}
    >>> def Foo(a, b=1):
    ...    print __dict__
    ...    
    >>> Foo(100)
    {'a': 100, 'b': 1}
    >>> class Test:
    ...    def Get(self):
    ...        print __dict__
    ...    
    >>> Test().Get()
    {'self': <__main__.Test instance at 0x0140DDA0>}
    This definitely works and, as far as I know, safe.

    Enjoy!

    Derrick
    Last edited by p4j; September 8th, 2004 at 10:20 AM. Reason: Improve dict wrapper
  20. #26
  21. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2004
    Location
    Toronto
    Posts
    3
    Rep Power
    0
    If you're going to be building a string iteratively (in a loop, for example), then instead of using string concatenation each time, the most efficient way of doing it is actually aggregating the component strings into a list and then joining them. So, for example:

    Code:

    somedict = {1: 'a', 2: 'b', 3: 'c', 4: 'd'}
    L = []
    for (k,v) in somedict:
    L.append('%s (%s)' % (v, k))

    results = ' '.join(L)
    Or if you're feeling more functional (should be similar efficiency):

    Code:
    somedict = {1: 'a', 2: 'b', 3: 'c', 4: 'd'}
    result = reduce(lambda a,b: a+' '+b, ['%s (%s)'%(a,b) for a,b in somedict.items()])
  22. #27
  23. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Oct 2004
    Location
    Bombay, India
    Posts
    159
    Rep Power
    12
    Hi,
    Originally Posted by MasterChief
    I had an idea to make a thread where people can post tips, tricks, thing to inspire or motivate people, links, etc.
    Guess I'm violating "the rules" by posting the Pitfalls of Python ... anyway I go ahead..being a complete Python aficionado, I took it in the right spirit!

    Cld check up this link....

    http://zephyrfalcon.org/labs/python_pitfalls.html

    Rgds,
    Subha
  24. #28
  25. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2005
    Posts
    174
    Rep Power
    11

    look through string and os modules


    For newb's like me. Seems to me that time and time again I'm looking through the string and os modules. Get to know them. They are your friends.

    If you do a lot of file processing, knowing the os module (there's a lot) makes using the filesystem a breeze. If you ever have to work with string constants, look at the string module. Odds are it's already there. (ie: string.digits, string.punctuation etc)

    So for example, I was making a general utility to search all my source files for a phrase inside my code. (I forget where I put stuff right?) Checked the os module. Sure enough, I found os.walk, (os.path.walk < 2.3) which was immediately useful for my code. This is a recursive search, taking a walk down each tree branch from whatever starting point. I can't imagine how much time this has saved me. Sweet.

    Cheers
    sf2k
  26. #29
  27. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2004
    Location
    Bimingham,AL
    Posts
    68
    Rep Power
    11

    Creating Python extensions in C/C++ with SWIG and compiling them with MinGW gcc under


    Creating Python extensions in C/C++ with SWIG and compiling them with MinGW gcc under Windows



    This page is a cookbook for creating Python extensions in C/C++ under Windows with SWIG, distutils and gcc (MinGW version).

    I am using to the following versions

    1 . SWIG Version 1.3.24

    2 . Python 2.3.4

    3.mingw 3.0.0

    1. Get and install MinGW gcc
    Download the compiler from http://www.mingw.org. This GCC compiler runs under Windows and compiled programs do not require support DLL like CygWin GCC.

    You only need to download MinGW-1.1.tar.gz (roughly 10,6 Mb). It contains the whole compiler, support utilities, documentation, librairies and header files.

    Once decompressed, you should add the \bin directory of MinGW to your path environment variable.
    (Example : Under Windows 95/98/ME, if you installed MinGW to c:\gcc, you would add SET PATH=c:\gcc\bin;%PATH% to your AUTOEXEC.BAT.)

    If installed properly, you should be able to run gcc --version anywhere. (Mine displays : 2.95.3-6).

    2 . Get and install Python
    Download the executable from www.python.org . As usually set the path to run python from anywhere.

    3 . Create libpython23.a
    To create Python extensions, you need to link against the Python library. Unfortunately, most Python distributions are provided with Python23.lib, a library in Microsoft Visual C++ format. GCC expects a .a file (libpython23.a to be precise.). Here's how to convert python23.lib to libpython23.a:

    Download pexport (from here or http://starship.python.net/crew/kernr/mingw32/pexports-0.42h.zip).
    Get Python23.dll (it should be somewhere on your harddrive).
    Run : pexports python23.dll > python23.def
    This will extract all symbols from python23.dll and write them into python23.def.
    Run : dlltool --dllname python23.dll --def python23.def --output-lib libpython23.a
    This will create libpython23.a (dlltool is part of MinGW utilities).
    Copy libpython23.a to c:\python23\libs\ (in the same directory as python23.lib).
    This trick should work for all Python versions, including future releases of Python. You can also use this trick to convert other libraries.


    4. Get and install SWIG
    SWIG is a wrapper for C/C++ sources. It allows you to use C/C++ functions and classes to Python with a minimum effort.

    Download SWIG binaries for Window ( Swigwin ) from http://www.swig.org, decompress them and add swig directory to your path (the directory where swig.exe is located).

    5 . Write .i files
    Write the .i files corresponding to your c++ files ( for more on how to write these files visit www.swig.org )



    6 . Use the following commands to get it going


    A . swig –python –c++ example.i

    output: example.py & example_wrap.cxx
    B. g++ -c *.c++

    output: .o files corresponding to your C++
    C. g++ -c example_wrap.cxx -Ic:\python23\include

    output: example_wrap.o
    D. g++ -shared *.o -o _example.pyd -Lc:\python23\libs -lpython23

    output: _example.pyd




    Hopefully this worked . Now you can do an “ import example “ in python .



    Author : Suryaprakash Gaddipati

    Contact : sprakash@uab.edu
  28. #30
  29. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69

    Making Python programs double clickable on Mac OS X.


    When I switched over to OS X I spent ages searching Google for a way to make my Python scripts run when I clicked them; like they do on Windows. It didn't seem like it was possible – no had an answer. Or if they did they weren't telling .

    Anyway, the answer's here finally for anyone who might be interested. Before warned: it takes a little work in the Terminal since you have to turn on the x bit on the files permissions to make the script executable!

    For this post I've created a very simple Hello World program on my Desktop called "test.py", which looks like this:

    Code:
    #!/usr/bin/env python
    
    print 'hello world!'
    Note: I am using the default shell on Mac OS Panther – bash. For more information about any of the commands used here type "man command".

    The first thing you need to do is to move to the directory that the Python file is in, this is done using the "cd" command. In our example our script is on the Desktop so lets cd there.

    Code:
    Mark-Smiths-Computer:~ Mark$ cd Desktop/
    Mark-Smiths-Computer:~/Desktop Mark$ ls -l
    total 8
    -rwxr--r--  1 Mark  Mark  43 18 Feb 19:14 test.py
    Next you need to give the script execution permissions (+x). You can do this using "chmod" command in one of two ways; for simplicity we're going with the easiest to understand:

    Code:
    Mark-Smiths-Computer:~/Desktop Mark$ chmod +x test.py
    Mark-Smiths-Computer:~/Desktop Mark$ ls -l
    total 8
    -rwxr-xr-x  1 Mark  Mark  43 18 Feb 19:14 test.py
    Mark-Smiths-Computer:~/Desktop Mark$
    Lastly you need to give your script the prefix, ".command". This simply tells OS X to open the Terminal execute the script. To rename out file we'll use the "mv" command:

    Note: You need to make sure that the shebang at the top of your program is correct, or your program wont run!

    Code:
    Mark-Smiths-Computer:~/Desktop Mark$ mv test.py test.py.command
    Mark-Smiths-Computer:~/Desktop Mark$ ls -l
    total 8
    -rwxr-xr-x  1 Mark  Mark  43 18 Feb 19:14 test.py.command
    Mark-Smiths-Computer:~/Desktop Mark$
    Now when you go back to your Desktop you should see a – and lets be fair – rather ugly looking Icon. If you double click it you will see the program open and run in a new Terminal Window.

    Conclusion:

    If your comfortable with the steps involved to do this then you're probably happy enough running your Python programs from the command line – the best way IMO. However if other people use your system then it could be very handy. In any case this should save some typing in the long run .

    Note: that this technique will work for any scripting language on OS X, including perl.

    Wa'la, one clickable Python script under OS X. Now if only they could be run from inside the scripts menu we'd really have something to play with!

    Enjoy,

    Mark.

    Comments on this post

    • SimonGreenhill agrees : Thanks Mark - I've been trying to do this for a while, but missed the .command bit.
    • codergeek42 agrees
    programming language development: www.netytan.com – Hula


IMN logo majestic logo threadwatch logo seochat tools logo