#1
  1. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,402
    Rep Power
    0

    Question Encodign issue in Python 3.3.1


    This is the code that although correct becaus it works with englisg(standARD ASCII letters) it wont with Greek:

    Code:
    if( log ): 
            name = log 
            # print specific client header info 
            cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ) 
            data = cur.fetchone()
    =======================

    The following is the live output of: tail -F /usr/local/apache/logs/error_log &

    Code:
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Error in sys.excepthook:, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'ascii' codec can't encode characters in position 2050-2056: ordinal not in range(128), referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] , referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Original exception was:, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Traceback (most recent call last):, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]   File "pelatologio.py", line 122, in <module>, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]     cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ), referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]   File "/opt/python3/lib/python3.3/site-packages/pymysql/cursors.py", line 108, in execute, referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11]     query = query.encode(charset), referer: http://superhost.gr/cgi-bin/pelatologio.py 
    [Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'latin-1' codec can't encode characters in position 46-52: ordinal not in range(256), referer: http://superhost.gr/cgi-bin/pelatologio.py

    I can understand that this is an encoding issue but i dont knwo how to fix this.
    please help.

    The search works only for english letters as expected but my cleints have Greek name and surnames.
  2. #2
  3. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Tallahassee
    Posts
    4
    Rep Power
    0

    Hope this helps you


    Somewhere you have a plain string encoded as "Windows-1252" (or "cp1252") containing a "RIGHT SINGLE QUOTATION MARK" () instead of an APOSTROPHE ('). This could come from a file you read, or even in a Python source file of yours; you could be running Python 2.x and have a # -*- coding: utf8 -*- line somewhere near the script's beginning, or you could be running Python 3.x.

    You don't give enough data; however, somewhere you have a cp1252-encoded string, which you try (explicitly or implicitly) to decode to unicode as utf-8. This won't work.

    Give us more info, and we'll try again to help you.
  4. #3
  5. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,402
    Rep Power
    0
    http://superhost.gr/?page=pelatologio.py works partially but only displays info when i click on person names in English, not with Greek names.

    If someone want to give me a hand with this, please do.

    if you cant see the webpahe due to http auth plz ask me to provide you user and pass.

    > But iu have it set up for 'utf-8' as seen in this statement.
    >
    > con = pymysql.connect( db = 'metrites', host = 'localhost', user =
    > 'me', passwd = 'somepass', charset='utf-8', init_command='SET NAMES UTF8' )

    That might not help... see below.

    >
    > Yoiu mean i shoudl chnag eit to greek isoo= (iso-8859-7)
    > but then i store english names and greek names as well, so it has to be a utf-8.

    Right. UTF-8 is the encoding you want.

    I think your problem is that when the MySQL database was created, it was
    not created using the UTF-8 mode, but rather latin-1 (despite your
    init_command which has nothing to do with database creation). You'll
    want to recreate the database and make double sure you've selected the
    right character set and collation:

    http://dev.mysql.com/doc/refman/5.0/en/charset-applications.html

    At least that should be your first step in debugging this problem.

    If the database is indeed utf-8, then you can move on to debugging why
    the python code is choosing latin-1 to encode your query string.

    I chekded the database and it is indeed UTF-8.
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Tallahassee
    Posts
    4
    Rep Power
    0

    Correct


    Enhance it and fix it by checking what you did wrong carefully. Search through the Internet even with your concern.
  8. #5
  9. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,402
    Rep Power
    0
    i did, ima searchign these 2 days now and i cant fix it.

IMN logo majestic logo threadwatch logo seochat tools logo