#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    42
    Rep Power
    3

    Remove/Skip some words from JSON's data


    Hi, I'm trying to find a solution to Skip or Remove some words from the data I got which is from JSON's files. I have a data like this:

    aa = results[16]['message'], results[17]['message']

    u'some headline text. http://aurlhere.com/somearticle/foryou \u2026', u'Another headline!\n\nhttp://www.somesite.com/somearticle.htm'

    As you can see in my file, there's \u2026 which contains with backslash(\). The problem is when I tried to insert these data to mysql, it shows me an error because It can not come across the '\' line. Is there any idea to remove or skip this specific words? Please help me find out.
    Last edited by ManiacDan; February 6th, 2013 at 01:05 PM. Reason: Removing overly political headlines.
  2. #2
  3. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,997
    Rep Power
    481

    Filter retains ASCII


    possible python2 solution demonstration
    Code:
    >>> a=u'Denver ߪ'
    >>> a
    u'Denver \u07ea'
    >>> print([ord(x)for x in a])
    >>> print(''.join(c for c in a if ord(c) < 256))
    Denver 
    >>>
    [code]Code tags[/code] are essential for python code and Makefiles!
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    42
    Rep Power
    3
    Thank you so much for your reply it helps me lots. It works like you said but when I tried it with multiple data like

    Code:
    a = []
    
    for i in range(10):
          a.append(results[i]['message'])
    
    a = (''.join(c for c in a if ord(c) < 256))
    It shows me an error :

    a = (''.join(c for c in a if ord(c) < 256))
    TypeError: ord() expected a character, but string of length 1245 found

    Do you know the way to solve this solution at a time? because I want to fetch multiple data and convert it to a properly format before I insert into SQL. Anyway, I appreciate your answer so much, thank you
  6. #4
  7. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,997
    Rep Power
    481
    Since you're trying to write in python, learn about the various object containers. dict , list , set , frozenset , tuple , str , and all those I've overlooked like bytes , unicode .

    Obviously, that is, obviously to a python programmer, you have a data structure that is more complicated than a unicode string whereas the implementation of the algorithmic idea I provided will not work with nested structures. Well, I won't count ['a','b'] as a nested structure.
    [code]Code tags[/code] are essential for python code and Makefiles!

IMN logo majestic logo threadwatch logo seochat tools logo