#1
  1. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,285
    Rep Power
    13

    Question Encodign issue in Python 3.3.1


    This is the code that although correct becaus it works with englisg(standARD ASCII letters) it wont with Greek:

    Code:
    if( log ): 
            name = log 
            # print specific client header info 
            cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ) 
            data = cur.fetchone()
    =======================

    The following is the live output of: tail -F /usr/local/apache/logs/error_log &

    Code:
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Error in sys.excepthook:, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'ascii' codec can't encode characters in position 2050-2056: ordinal not in range(128), referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] , referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Original exception was:, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Traceback (most recent call last):, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]   File "pelatologio.py", line 122, in <module>, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]     cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ), referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]   File "/opt/python3/lib/python3.3/site-packages/pymysql/cursors.py", line 108, in execute, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]     query = query.encode(charset), referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'latin-1' codec can't encode characters in position 46-52: ordinal not in range(256), referer: http://superhost.gr/cgi-bin/pelatologio.py

    I can understand that this is an encoding issue but i dont knwo how to fix this.
    please help.

    The search works only for english letters as expected but my cleints have Greek name and surnames.
    What is now proved was once only imagined!
  2. #2
  3. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Tallahassee
    Posts
    5
    Rep Power
    0

    Hope this helps you


    Somewhere you have a plain string encoded as "Windows-1252" (or "cp1252") containing a "RIGHT SINGLE QUOTATION MARK" () instead of an APOSTROPHE ('). This could come from a file you read, or even in a Python source file of yours; you could be running Python 2.x and have a # -*- coding: utf8 -*- line somewhere near the script's beginning, or you could be running Python 3.x.

    You don't give enough data; however, somewhere you have a cp1252-encoded string, which you try (explicitly or implicitly) to decode to unicode as utf-8. This won't work.

    Give us more info, and we'll try again to help you.
  4. #3
  5. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,285
    Rep Power
    13
    http://superhost.gr/?page=pelatologio.py works partially but only displays info when i click on person names in English, not with Greek names.

    If someone want to give me a hand with this, please do.

    if you cant see the webpahe due to http auth plz ask me to provide you user and pass.

    > But iu have it set up for 'utf-8' as seen in this statement.
    >
    > con = pymysql.connect( db = 'metrites', host = 'localhost', user =
    > 'me', passwd = 'somepass', charset='utf-8', init_command='SET NAMES UTF8' )

    That might not help... see below.

    >
    > Yoiu mean i shoudl chnag eit to greek isoo= (iso-8859-7)
    > but then i store english names and greek names as well, so it has to be a utf-8.

    Right. UTF-8 is the encoding you want.

    I think your problem is that when the MySQL database was created, it was
    not created using the UTF-8 mode, but rather latin-1 (despite your
    init_command which has nothing to do with database creation). You'll
    want to recreate the database and make double sure you've selected the
    right character set and collation:

    http://dev.mysql.com/doc/refman/5.0/en/charset-applications.html

    At least that should be your first step in debugging this problem.

    If the database is indeed utf-8, then you can move on to debugging why
    the python code is choosing latin-1 to encode your query string.

    I chekded the database and it is indeed UTF-8.
    What is now proved was once only imagined!
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Tallahassee
    Posts
    5
    Rep Power
    0

    Correct


    Enhance it and fix it by checking what you did wrong carefully. Search through the Internet even with your concern.
  8. #5
  9. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,285
    Rep Power
    13
    i did, ima searchign these 2 days now and i cant fix it.
    What is now proved was once only imagined!

IMN logo majestic logo threadwatch logo seochat tools logo