#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    64
    Rep Power
    12

    Python and SQLite3


    I've tried to figure this out all day.

    I have a CGI script when run from the web, I get a 500 error. The error log reveals nothing.
    Now, when I run the script from the command line it works perfectly! All the correct headers are there too.

    I've tracked it down to this bit of code.

    login.psp
    Code:
    functions.newSession(sid,data)
    functions.py
    Code:
         45 def newSession(sid,data):
         46         import database, config, cPickle
         47 
         48         # data dictionary format: 
         49         # data['uid']
         50         # data['phash']
         51         # data['lastvisit']
         52 
         53         # Add sid to dictionary 
         54         data['sid'] = sid
         55         # Pickle that dictionary!
         56         encdata = cPickle.dumps(data)
         57 
         58         conn = database.database(dbtype=config.dbb['dbtype'],database=config.dbb['database'])
         59         cu = conn.givecursor()
         60 
         61         # Insert the pickled dictionary into the db
         62         cu.execute("INSERT INTO sessions VALUES(\"" + sid + "\", \"" + encdata + "\")")
         63         conn.close()
    This code worked perfectly before with pysqlite and sqlite2.
    It only stopped working when I began to migrate to pysqlite3 and sqlite3.
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2003
    Location
    Norway
    Posts
    41
    Rep Power
    12
    Are you using mod_python's Python Server Pages? I mean since your file is called login.psp. If you are running CGI, try to add
    Code:
    import cgtb
    cgitb.enable()
    at the top of your script. That should print out a complete traceback so you can see what's happening. It's a lot easier to figure out what's wrong if you figure out exactly which line the error occurs in.

    If you change your query execution to something like this, characters like ' will be automatically escaped so you avoid that causing sql syntax errors (and sql injection attacks).
    Code:
    cu.execute("INSERT INTO sessions VALUES '%s', '%s'"%( sid,encdata))
    It's also better and faster to store the session data directly to the sqlite db instead of pickle, store and then later fetch data and unpickle. Waste of space (pickles are larger) and cpu (pickle/unpickle operations).
    Good web hosting info - articles about web hosting
    hb's web dev blog
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    64
    Rep Power
    12
    I'm not using mod_python. I'm just using .psp so I can tell my content pages from my modules.

    I'm using cgitb already.
    And I'll try it that way.

    Edit: retried using

    Code:
    cu.execute("INSERT INTO sessions VALUES('%s', '%s')"%(sid,encdata))
    And it does the same thing.
  6. #4
  7. Mini me.
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2003
    Location
    Cambridge, UK
    Posts
    783
    Rep Power
    13
    ZeRo,

    Just a few thougts:
    Does the function work when it is used in an ordinary python module?
    Do you always get a database connection - line 58?
    Do you always get a cursor - line 59?
    What happens if you don't?

    grim
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    64
    Rep Power
    12
    I'm not really sure. But I know for a fact it works when called from the command with

    Code:
    python ./login.psp Submit=1\&uid=1\&pass=password
    The data is inserted into the database and the correct HTML and headers are printed...
  10. #6
  11. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    What does Apache say to this? I've never heard of Python bail without an error - unless Python is crashing staight out. In which case it wouldn't matter if It was run under the command line or the server, both would have the same result.

    I'd take another look at the Error logs. There has to be some little clue there.

    Also should it not be:
    Code:
    python ./login.psp Submit=1&uid=1&pass=password
    Since there shouldn't be any spaces in the query string. But I find it a little odd that Python is accepting this... input from the command line comes though sys.argv and usually, input from a webpage is gathered by cgi.FieldStorage() and the two do not exactly play well .

    So, I'd have to say attach your full program!

    Mark.
    programming language development: www.netytan.com Hula

  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    64
    Rep Power
    12
    All the error log says is:

    Code:
    [Thu Sep 23 06:15:21 2004] [error] [client 192.168.1.252] Premature end of script headers: login.psp, referer: http://gmeow.com/bbs/login.psp
    login.psp
    Code:
    #!/usr/bin/python
    
    import functions, Cookie, cgi, template, config, database, time
    from sys import exit
    from os import environ
    import cgitb; cgitb.enable()
    
    te = template.dbb_template(config.dbb['skin'])
    
    form = cgi.FieldStorage()
    if not form.has_key("Submit"):	
    	C = Cookie.SimpleCookie()
    	try:
    		functions.texthtml()
    		C.load(environ['HTTP_COOKIE'])
    		te.loadtemplate('header')
    		te.printtemplate()
    
    		sid = C['dbbcf_sid'].value
    		uid = C['dbbcf_uid'].value
    
    		print "User " + str(uid) + " already logged in!"
    		
    		te.loadtemplate('footer')
    		te.printtemplate()
    		
    		exit()
    	except KeyError:
    		te.loadtemplate('login_form')
    		te.printtemplate()
    
    		te.loadtemplate('footer')
    		te.printtemplate()
    
    		exit()
    
    if form['uid'].value == None:
    	te.loadtemplate('header')
    	te.printtemplate()
    	functions.throwerror("Please enter a uid.")
    if form['pass'].value == None:
    	te.loadtemplate('header')
    	te.printtemplate()
    	functions.throwerror("Please enter a pass.")
    
    import md5
    # First pass...
    p = md5.new(form['pass'].value)
    p = p.hexdigest()
    # Second pass...
    p = md5.new(p)
    p = p.hexdigest()
    
    conn = database.database(dbtype=config.dbb['dbtype'],database=config.dbb['database'])
    cu = conn.givecursor()
    
    cu.execute("SELECT password FROM users WHERE id=" + str(form['uid'].value))
    result = cu.fetchone()
    conn.close()
    
    p2 = md5.new(result[0])
    p2 = p2.hexdigest()
    
    if p != p2:
    	functions.texthtml()
    	te.loadtemplate('header')
    	te.printtemplate()
    	functions.throwerror("Incorrect password!")
    
    sid = functions.genSessionId(environ['REMOTE_ADDR'])
    
    C = Cookie.SimpleCookie()
    C['dbbcf_uid'] = form['uid'].value
    C['dbbcf_sid'] = sid
    
    print C
    
    data = {}
    data['uid'] = form['uid'].value
    data['phash'] = p
    data['lastvisit'] = time.time()
    
    # Create a new session
    functions.newSession(sid,data)
    
    functions.texthtml()
    
    te.loadtemplate('header')
    te.printtemplate()
    
    print "<a href='index.psp?sid=" + sid + "'>Back to home</a>"
    
    te.loadtemplate('footer')
    te.printtemplate()
    functions.py
    Code:
    def texthtml():
    	print "Content-Type: text/html\n\n\n"
    
    def throwerror(error):
    	import template,config,sys
    	te = template.dbb_template(config.dbb['skin'])
    
    	te.loadtemplate("error")
    	te.replace("#ERROR#",error)
    	te.printtemplate()
    	
    	te.loadtemplate("footer")
    	te.printtemplate()
    	
    	sys.exit()
    def escapeRegex(toesc):
    	import sre
    
    	toesc = sre.sub("\(","\\(",toesc)
    	toesc = sre.sub("\)","\\)",toesc)
    	toesc = sre.sub("\;","\\;",toesc)
    	toesc = sre.sub("\|","\\|",toesc)
    	toesc = sre.sub("\.","\\.",toesc)
    	toesc = sre.sub("\^","\\^",toesc)
    	toesc = sre.sub("\$","\\$",toesc)
    	toesc = sre.sub("\*","\\*",toesc)
    	toesc = sre.sub("\+","\\+",toesc)
    	toesc = sre.sub("\?","\\?",toesc)
    	toesc = sre.sub("\{","\\{",toesc)
    	toesc = sre.sub("\}","\\}",toesc)
    	toesc = sre.sub("\[","\\[",toesc)
    	toesc = sre.sub("\]","\\]",toesc)
    
    	return toesc
    
    # Session stuff #
    def genSessionId(rdat):
    	import md5, time, base64, string
    	m = md5.new()
    	m.update('ababa was here. ~DogBB~ ababa was here.')
    	m.update(str(time.time()))
    	m.update(str(rdat))
    	return string.replace(base64.encodestring(m.digest())[:-3], '/', '$')
    
    def newSession(sid,data):
    	import database, config, cPickle
    
    	# data dictionary format: 
    	# data['uid']
    	# data['phash']
    	# data['lastvisit']
    	
    	# Add sid to dictionary 
    	data['sid'] = sid
    	# Pickle that dictionary!
    	encdata = cPickle.dumps(data)
    	
    	conn = database.database(dbtype=config.dbb['dbtype'],database=config.dbb['database'])
    	cu = conn.givecursor()
    
    	# Insert the pickled dictionary into the db
    	cu.execute("INSERT INTO sessions VALUES('%s', '%s')"%(sid,encdata))
    	conn.close()
    def readSession(sid):
    	import database, config, cPickle
    	
    	conn = database.database(dbtype=config.dbb['dbtype'],database=config.dbb['database'])
    	cu = conn.givecursor()
    
    	# Get the pickle! =B
    	cu.execute("SELECT encdata FROM sessions WHERE sid='" + sid + "' LIMIT 1")
    	result = cu.fetchone()
    	data = result[0]
    	data = cPickle.loads(data)	
    	return data
  14. #8
  15. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Your problem is linked to the MIME header you are or are not sending. As a rule with CGI programming - there are exceptions - the headers should be outputted directly after importing any desired module.

    I think thats your problem. Move your call to texthtml() to the top of the page and you should be groovin' .

    Hope this helps,

    Mark.
    programming language development: www.netytan.com Hula

  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2003
    Posts
    64
    Rep Power
    12
    If I move it to the top of the page then my cookies won't work. The need to printed before the Content-Type.
  18. #10
  19. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    I cant go through your whole code and rewrite it for you . Theres a lot of things in there that should be changed i.e. importing inside functions/classes isn't the best idea. All I'm saying is that something else is getting though to stuff up the headers acording to Apache.

    Edit: Two nice tricks to working with this kind of thing are to look very closly at the command lines output; more useful being to put a content type header at the very top of your page and look at the pages source.

    Give it a go,

    Mark.
    Last edited by netytan; September 25th, 2004 at 12:30 AM.
    programming language development: www.netytan.com Hula


IMN logo majestic logo threadwatch logo seochat tools logo