Page 2 - Discuss Tips, tricks, inspiration, etc in the Python Programming forum on Dev Shed. Tips, tricks, inspiration, etc Python Programming forum discussing coding techniques, tips and tricks, and Zope related information. Python was designed from the ground up to be a completely object-oriented programming language.
Posts: 3
Time spent in forums: < 1 sec
Reputation Power: 0
Performance tweaks
Hi,
just a note about performance: if you have the choice between code readability and performance, it's usually better to chose readability, unless you are really certain you are inside a hotspot of your code. If you want to check if your code tuning really did change execution time, use the timer module. If you post performance tips here, a sample timer output can be helpful.
Code:
import timing
def f():
timing.start()
x=[]
for i in xrange(1,10000):
x.append(str(i))
s = ''.join(x)
timing.finish()
return timing.milli()
def f2():
timing.start()
x=''
for i in xrange(1, 10000):
x += str(i)
timing.finish()
return timing.milli()
print "Creating string with append: %sms" % f2()
print "Creating string with list.join: %sms" % f()
which looks like this (don't laugh about my old cpu: ):
Quote:
Creating string with append: 225ms
Creating string with list.join: 58ms
Posts: 2,537
Time spent in forums: 1 Week 2 Days 18 h 17 m 47 sec
Reputation Power: 68
Maybe this is a Windows thing but there doesn't appear to be a 'timer' or 'timing' module in the standard library. If this is a third party module maby you could post a link to it's homepage .
So for those of you out there who are scratching there heads and saying "ok, now we know why that didnt work... but how do we check preformance ", 'timeit'.
Pythons timeit module (new in Python 2.3 - consider using Pystone or one of the various preformance tracking modules to check preformance with earlier Python versions) makes checking code snippets surprisingly easy!
Posts: 3
Time spent in forums: < 1 sec
Reputation Power: 0
Oops
Quote:
Originally Posted by netytan
Maybe this is a Windows thing but there doesn't appear to be a 'timer' or 'timing' module in the standard library.
Yes, I'm sorry, I didn't expect to find modules in my installation that weren't part of the standard lib. Somehow my distribution installed a C-Style library with python bindings called timing, perhaps it's part of the C-profiler for python under linux, don't know. Glad you catched this before I cluttered too many files with it, would have been portability nightmare .
Next time I find something using help() I'll check with the docs if it really is a standard module. That's a nice tip, too .
Posts: 783
Time spent in forums: 3 Days 2 h 15 m 57 sec
Reputation Power: 12
Posting Code
If you want help - it is a lot easier for people to read your code if you use the # button on the submit form to make a code block that preserves the layout.
Posts: 2,537
Time spent in forums: 1 Week 2 Days 18 h 17 m 47 sec
Reputation Power: 68
Another nice piece of functionality (related to Grims post) is that you can highlight a section of text in you're post and hit pretty much any of the buttons and the respective tags are placed around the highlighted area!
Posts: 1
Time spent in forums: < 1 sec
Reputation Power: 0
A little tip
Tip: During the active shell console in python, you can ask python for documentation as follows:
Code:
# This is a fake function
def myFakeFcn():
""" These are the commented
documents of my function named
myFakeFcn() where spacing is easily
preserved !"""
print "This function has no purpose!!",
print myFakeFcn.__doc__ #this will output the documentation
>>>
These are the commented
documents of my function named
myFakeFcn() where spacing is easily
preserved !
I know another post stated that the __doc__ could be used as a reference, but I wanted to point out that it can be used during an active shell.
Posts: 2,537
Time spent in forums: 1 Week 2 Days 18 h 17 m 47 sec
Reputation Power: 68
This is a commonly used method of adding test code directly into your modules or even telling your program what to do when its run. In any case very usfull stuff... "But we already know this?!"... Most do but as it seems to come up periodically so i thought its worth mentioning here .
Code:
#!/usr/bin/env python
#Any code that you want to be run when Python interprets
#the file. Imported or run.
if __name__ == '__main__':
#This part of the program will only be run if the program
#if executed (is running). If the program is imported then
#Python will ignore this part of the program =].
Note: this isnt a must i.e. main in C/C++. Its just a nice way to organise your programs.
Link in detail: Recipe of the day. Every day, a recipe from the Python Cookbook is featured.
I just purchased Python Cookbook (edited by Alex Martelli & David Ascher). By the way, I highly recommend this book.
Here is what's on the back of the book:
"The Python Cookbook is a collection of problems, solutions, and practical examples for Python programmers, written by Python programmers. Over the past year, members of the Python community have contributed material to an online repository of Python recipes hosted by ActiveState. This book contains the best of those recipes, accompanied by overviews and background material by key Python figures.
The recipes in the Python Cookbook range from simple tasks, such as working with dictionaries and list comprehensions, to entire modules that demonstrate complex tasks, such as a templating system and network monitoring. This book contains over 200 recipes.
This book is a treasure trove of useful code for all Python programmers, from novices to advanced practioners, with contributions from such Python luminaries as Guido van Rossum, David Ascher, Tim Peters, Paul Prescod, Mark Hammond, and Alex Martelli, as well as over 100 other Python programmers. The recipes highlight Python best practices and can be used directly in day-to-day programming tasks, as a source of ideas, or as a way to learn more about Python.
The recipes in the Python Cookbook were edited by David Ascher, who is on the board of the Python Software Foundation and is the co-author of Learning Python, and Alex Martelli, who is known for his numerous and exhaustive postings on the Python mailing list. The book contains a foreword by Guido van Rossum, the creator of Python."
Posts: 72
Time spent in forums: 2 h 47 m 23 sec
Reputation Power: 10
I thought that this would benefit those who are like me learning python and need a little help in understanding what is going on with their code. You can walk step by step and see what is happening line by line, this is similar to debugging in VB. Hope someone finds it useful in their learning proccess.
Posts: 27
Time spent in forums: 2 h 37 m 17 sec
Reputation Power: 0
Creating a Local __dict__
Accessing the local namespace dict is already done by calling the builtin method locals(). However, the limitation of doing this is that modifying the dict returned by locals() is not considered 'safe'. Safe in the sense that any modifications are not guaranteed to be reflected in the actual namespace.
Another approach I have taken is to access the local dict via:
Code:
sys._getframe().f_locals
From all of the documentation I've read and input I've received, the f_locals seems 'safe' for modification.
Using this approach, I have created a psuedo local __dict__. Here is a snippet of the main script code to do this:
Code:
import sys
__all__ = ['local_dict']
bad = ['__init__', '__new__', '__repr__']
############################################################
class __dict( dict ):
"""
Wrapper to mimic a local dict.
Written by Derrick Wallace
"""
def __init__( self, *args ):
dict.__init__( self, *args )
for attr in dict.__dict__:
if callable( dict.__dict__[attr] ) and ( not attr in bad ):
exec( 'def %s(self, *args): return dict.%s(sys._getframe(1).f_locals, *args)'%( attr, attr ) )
exec( '__dict.%s = %s'%( attr, attr ) )
#####################################
def __repr__( self, *args ):
# Must implement repr to prevent recursion
if sys._getframe(1).f_code == sys._getframe().f_code:
return '{...}'
return dict.__repr__( sys._getframe(1).f_locals, *args )
local_dict = __dict()
I've only 'wrapped' some basic dict methods here, but all of them should and can be wrapped. Notice the repr method is different in that it must perform a check, this is to prevent recursion. In order to finish implementing this as a psuedo local __dict__ in another script, it must be imported. Assuming the above script is saved in a file called local_test.py, then this is how it could be done:
Code:
from local_test import local_dict as __dict__
Now __dict__ acts like the local __dict__ for ANY namespace, whether it be a class method or a function. I just think that is cool! For example:
Posts: 3
Time spent in forums: 32 m 40 sec
Reputation Power: 0
Quote:
If you're going to be building a string iteratively (in a loop, for example), then instead of using string concatenation each time, the most efficient way of doing it is actually aggregating the component strings into a list and then joining them. So, for example:
Code:
somedict = {1: 'a', 2: 'b', 3: 'c', 4: 'd'}
L = []
for (k,v) in somedict:
L.append('%s (%s)' % (v, k))
results = ' '.join(L)
Or if you're feeling more functional (should be similar efficiency):
Code:
somedict = {1: 'a', 2: 'b', 3: 'c', 4: 'd'}
result = reduce(lambda a,b: a+' '+b, ['%s (%s)'%(a,b) for a,b in somedict.items()])
Posts: 159
Time spent in forums: 43 m 45 sec
Reputation Power: 11
Hi,
Quote:
Originally Posted by MasterChief
I had an idea to make a thread where people can post tips, tricks, thing to inspire or motivate people, links, etc.
Guess I'm violating "the rules" by posting the Pitfalls of Python ... anyway I go ahead..being a complete Python aficionado, I took it in the right spirit!
Posts: 174
Time spent in forums: 2 Days 18 h 23 m 57 sec
Reputation Power: 10
look through string and os modules
For newb's like me. Seems to me that time and time again I'm looking through the string and os modules. Get to know them. They are your friends.
If you do a lot of file processing, knowing the os module (there's a lot) makes using the filesystem a breeze. If you ever have to work with string constants, look at the string module. Odds are it's already there. (ie: string.digits, string.punctuation etc)
So for example, I was making a general utility to search all my source files for a phrase inside my code. (I forget where I put stuff right?) Checked the os module. Sure enough, I found os.walk, (os.path.walk < 2.3) which was immediately useful for my code. This is a recursive search, taking a walk down each tree branch from whatever starting point. I can't imagine how much time this has saved me. Sweet.
Posts: 68
Time spent in forums: 14 h 58 m 22 sec
Reputation Power: 9
Creating Python extensions in C/C++ with SWIG and compiling them with MinGW gcc under
Creating Python extensions in C/C++ with SWIG and compiling them with MinGW gcc under Windows
This page is a cookbook for creating Python extensions in C/C++ under Windows with SWIG, distutils and gcc (MinGW version).
I am using to the following versions
1 . SWIG Version 1.3.24
2 . Python 2.3.4
3.mingw 3.0.0
1. Get and install MinGW gcc
Download the compiler from http://www.mingw.org. This GCC compiler runs under Windows and compiled programs do not require support DLL like CygWin GCC.
You only need to download MinGW-1.1.tar.gz (roughly 10,6 Mb). It contains the whole compiler, support utilities, documentation, librairies and header files.
Once decompressed, you should add the \bin directory of MinGW to your path environment variable.
(Example : Under Windows 95/98/ME, if you installed MinGW to c:\gcc, you would add SET PATH=c:\gcc\bin;%PATH% to your AUTOEXEC.BAT.)
If installed properly, you should be able to run gcc --version anywhere. (Mine displays : 2.95.3-6).
2 . Get and install Python
Download the executable from www.python.org . As usually set the path to run python from anywhere.
3 . Create libpython23.a
To create Python extensions, you need to link against the Python library. Unfortunately, most Python distributions are provided with Python23.lib, a library in Microsoft Visual C++ format. GCC expects a .a file (libpython23.a to be precise.). Here's how to convert python23.lib to libpython23.a:
Download pexport (from here or http://starship.python.net/crew/kernr/mingw32/pexports-0.42h.zip).
Get Python23.dll (it should be somewhere on your harddrive).
Run : pexports python23.dll > python23.def
This will extract all symbols from python23.dll and write them into python23.def.
Run : dlltool --dllname python23.dll --def python23.def --output-lib libpython23.a
This will create libpython23.a (dlltool is part of MinGW utilities).
Copy libpython23.a to c:\python23\libs\ (in the same directory as python23.lib).
This trick should work for all Python versions, including future releases of Python. You can also use this trick to convert other libraries.
4. Get and install SWIG
SWIG is a wrapper for C/C++ sources. It allows you to use C/C++ functions and classes to Python with a minimum effort.
Download SWIG binaries for Window ( Swigwin ) from http://www.swig.org, decompress them and add swig directory to your path (the directory where swig.exe is located).
5 . Write .i files
Write the .i files corresponding to your c++ files ( for more on how to write these files visit www.swig.org )
6 . Use the following commands to get it going
A . swig python c++ example.i
output: example.py & example_wrap.cxx
B. g++ -c *.c++
output: .o files corresponding to your C++
C. g++ -c example_wrap.cxx -Ic:\python23\include
output: example_wrap.o
D. g++ -shared *.o -o _example.pyd -Lc:\python23\libs -lpython23
output: _example.pyd
Hopefully this worked . Now you can do an import example in python .
Posts: 2,537
Time spent in forums: 1 Week 2 Days 18 h 17 m 47 sec
Reputation Power: 68
Making Python programs double clickable on Mac OS X.
When I switched over to OS X I spent ages searching Google for a way to make my Python scripts run when I clicked them; like they do on Windows. It didn't seem like it was possible no had an answer. Or if they did they weren't telling .
Anyway, the answer's here finally for anyone who might be interested. Before warned: it takes a little work in the Terminal since you have to turn on the x bit on the files permissions to make the script executable!
For this post I've created a very simple Hello World program on my Desktop called "test.py", which looks like this:
Code:
#!/usr/bin/env python
print 'hello world!'
Note: I am using the default shell on Mac OS Panther bash. For more information about any of the commands used here type "man command".
The first thing you need to do is to move to the directory that the Python file is in, this is done using the "cd" command. In our example our script is on the Desktop so lets cd there.
Code:
Mark-Smiths-Computer:~ Mark$ cd Desktop/
Mark-Smiths-Computer:~/Desktop Mark$ ls -l
total 8
-rwxr--r-- 1 Mark Mark 43 18 Feb 19:14 test.py
Next you need to give the script execution permissions (+x). You can do this using "chmod" command in one of two ways; for simplicity we're going with the easiest to understand:
Code:
Mark-Smiths-Computer:~/Desktop Mark$ chmod +x test.py
Mark-Smiths-Computer:~/Desktop Mark$ ls -l
total 8
-rwxr-xr-x 1 Mark Mark 43 18 Feb 19:14 test.py
Mark-Smiths-Computer:~/Desktop Mark$
Lastly you need to give your script the prefix, ".command". This simply tells OS X to open the Terminal execute the script. To rename out file we'll use the "mv" command:
Note: You need to make sure that the shebang at the top of your program is correct, or your program wont run!
Code:
Mark-Smiths-Computer:~/Desktop Mark$ mv test.py test.py.command
Mark-Smiths-Computer:~/Desktop Mark$ ls -l
total 8
-rwxr-xr-x 1 Mark Mark 43 18 Feb 19:14 test.py.command
Mark-Smiths-Computer:~/Desktop Mark$
Now when you go back to your Desktop you should see a and lets be fair rather ugly looking Icon. If you double click it you will see the program open and run in a new Terminal Window.
Conclusion:
If your comfortable with the steps involved to do this then you're probably happy enough running your Python programs from the command line the best way IMO. However if other people use your system then it could be very handy. In any case this should save some typing in the long run .
Note: that this technique will work for any scripting language on OS X, including perl.
Wa'la, one clickable Python script under OS X. Now if only they could be run from inside the scripts menu we'd really have something to play with!