#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2015
    Posts
    35
    Rep Power
    4

    Pythonscript to read json data


    Hi Guys,
    When execute the below script to parse json getting error ,Please help .

    Code:
    import json
    path = 'c:/mohan/a.txt'
    records = [json.loads(line) for line in open(path, 'rb')]
    
    Traceback (most recent call last):
      File "<pyshell#157>", line 1, in <module>
        records = [json.loads(line) for line in open(path, 'r')]
      File "C:\Users\mohana.pr\Downloads\WinPython-64bit-2.7.9.3\python-2.7.9.amd64\lib\json\__init__.py", line 338, in loads
        return _default_decoder.decode(s)
      File "C:\Users\mohana.pr\Downloads\WinPython-64bit-2.7.9.3\python-2.7.9.amd64\lib\json\decoder.py", line 366, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "C:\Users\mohana.pr\Downloads\WinPython-64bit-2.7.9.3\python-2.7.9.amd64\lib\json\decoder.py", line 384, in raw_decode
        raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded
    Input
    Code:
    { "a": "Mozilla\\/5.0 (Windows NT 6.1; WOW64) AppleWebKit\
    
    \/535.11(KHTML, like Gecko) Chrome\\/17.0.963.78 Safari\
    
    \/535.11", "c": "US", "nk": 1,"tz": "America\\/New_York", "gr":
    
    "MA", "g": "A6qOVH", "h":"wfLQtf", "l":"orofrog", "al": "en-
    
    US,en;q=0.8", "hh": "1.usa.gov", "r":"http:\\/
    
    \/www.facebook.com\\/l\\/7AQEFzjSi\\/1.usa.gov\\/wfLQtf",
    
    "u":"http:\\/\\/www.ncbi.nlm.nih.gov\\/pubmed\\/22415991", "t":
    
    1331923247, "hc":1331822918, "cy": "Danvers", "ll": [ 42.576698,
    
    -70.954903 ] }
    Expected output
    Code:
    {u'a': u'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like
    Gecko) Chrome/17.0.963.78 Safari/535.11',
    u'al': u'en-US,en;q=0.8',
    u'c': u'US',
    u'cy': u'Danvers',
    u'g': u'A6qOVH',
    u'gr': u'MA',
    u'h': u'wfLQtf',
    u'hc': 1331822918,
    u'hh': u'1.usa.gov',
    u'l': u'orofrog',
    u'll': [42.576698, -70.954903],
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2007
    Location
    Joensuu, Finland
    Posts
    471
    Rep Power
    71
    I think you should first read in the data and only then use json.loads() to parse it, for instance:

    Code:
    data = ' '.join([line.strip() for line in open(path, 'r')])
    records = json.loads(data)
    Now you are trying to read and parse the file line-by-line even though the a single line does not contain valid JSON data.

    Comments on this post

    • Will-O-The-Wisp agrees
    My armada: Debian GNU/Linux 8 (desktop, home laptop, work laptop), Raspbian GNU/Linux 8 (nameserver), Ubuntu 14.04.3 LTS (HTPC), PC-BSD 10.2 (testbed), Android 4.2.1 (tablet)
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2015
    Posts
    35
    Rep Power
    4
    Originally Posted by SuperOscar
    I think you should first read in the data and only then use json.loads() to parse it, for instance:

    Code:
    data = ' '.join([line.strip() for line in open(path, 'r')])
    records = json.loads(data)
    Now you are trying to read and parse the file line-by-line even though the a single line does not contain valid JSON data.
    Thanks for your prompt reply .but still getiing the error when execute the below

    >>> import json
    >>> data = ' '.join([line.strip() for line in open('c:/mohan/c.txt, 'r')])

    SyntaxError: EOL while scanning string literal
  6. #4
  7. Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Aug 2011
    Posts
    5,975
    Rep Power
    510
    Code:
    '''
    where the data file is ./d
    
    this program displays
    
    {'a': 'Mozilla\\/5.0 (Windows NT 6.1; WOW64) AppleWebKit\\/535.11(KHTML, '
          'like Gecko) Chrome\\/17.0.963.78 Safari\\/535.11',
     'al': 'en-US,en;q=0.8',
     'c': 'US',
     'cy': 'Danvers',
     'g': 'A6qOVH',
     'gr': 'MA',
     'h': 'wfLQtf',
     'hc': 1331822918,
     'hh': '1.usa.gov',
     'l': 'orofrog',
     'll': [42.576698, -70.954903],
     'nk': 1,
     'r': 'http:\\//www.facebook.com\\/l\\/7AQEFzjSi\\/1.usa.gov\\/wfLQtf',
     't': 1331923247,
     'tz': 'America\\/New_York',
     'u': 'http:\\/\\/www.ncbi.nlm.nih.gov\\/pubmed\\/22415991'}
    '''
    
    import pprint
    import json
    data = ''.join(line.strip() for line in open('d', 'r'))
    pprint.pprint(json.loads(data))
    [code]Code tags[/code] are essential for python code and Makefiles!
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2015
    Posts
    35
    Rep Power
    4
    Originally Posted by b49P23TIvg
    Code:
    '''
    where the data file is ./d
    
    this program displays
    
    {'a': 'Mozilla\\/5.0 (Windows NT 6.1; WOW64) AppleWebKit\\/535.11(KHTML, '
          'like Gecko) Chrome\\/17.0.963.78 Safari\\/535.11',
     'al': 'en-US,en;q=0.8',
     'c': 'US',
     'cy': 'Danvers',
     'g': 'A6qOVH',
     'gr': 'MA',
     'h': 'wfLQtf',
     'hc': 1331822918,
     'hh': '1.usa.gov',
     'l': 'orofrog',
     'll': [42.576698, -70.954903],
     'nk': 1,
     'r': 'http:\\//www.facebook.com\\/l\\/7AQEFzjSi\\/1.usa.gov\\/wfLQtf',
     't': 1331923247,
     'tz': 'America\\/New_York',
     'u': 'http:\\/\\/www.ncbi.nlm.nih.gov\\/pubmed\\/22415991'}
    '''
    
    import pprint
    import json
    data = ''.join(line.strip() for line in open('d', 'r'))
    pprint.pprint(json.loads(data))
    Hi ,
    Recently downloaded data from twitter ,i am trying to use the same code but it throwing error "AttributeError: 'module' object has no attribute 'loads'"

    import pprint
    import json
    data = ''.join(line.strip() for line in open('c:/mohan/c.txt', 'r'))
    pprint.pprint(json.loads(data))

    Twitter data(c.txt)

    Code:
    {"results":[
     
         {"text":"@twitterapi  http://tinyurl.com/ctrefg",
     
         "to_user_id":396524,
     
         "to_user":"TwitterAPI",
     
         "from_user":"jkoum",
     
         "metadata":
     
         {
     
          "result_type":"popular",
     
          "recent_retweets": 109
     
      
     
         },
     
         "id":1478555574,   
     
         "from_user_id":1833773,
     
         "iso_language_code":"nl",
     
         "source":"<a href="http://twitter.com/">twitter< /a>",
     
         "profile_image_url":"http://s3.amazonaws.com/twitter_production/profile_images/118412707/2522215727_a5f07da155_b_normal.jpg",
     
         "created_at":"Wed, 08 Apr 2009 19:22:10 +0000"},
     
         ... truncated ...],
     
         "since_id":0,
     
         "max_id":1480307926,
     
         "refresh_url":"?since_id=1480307926&amp;q=%40twitterapi",
     
         "results_per_page":15,
     
         "next_page":"?page=2&amp;max_id=1480307926&amp;q=%40twitterapi",
     
         "completed_in":0.031704,
     
         "page":1,
     
         "query":"%40twitterapi"}
     
    }
    Thanks ,
    Mohan
  10. #6
  11. Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Aug 2011
    Posts
    5,975
    Rep Power
    510
    This data line looks invalid, and my python installations (versions 2 and 3) agree:
    Code:
         "source":"<a href="http://twitter.com/">twitter< /a>",
    I understand that python raises an Exception. I'm afraid that figuring out why you were told "attribute not in module" rather than
    ValueError: Expecting , delimiter: line 1 column 268 (char 267)
    exceeds my paylevel.
    [code]Code tags[/code] are essential for python code and Makefiles!
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Apr 2015
    Posts
    35
    Rep Power
    4
    Originally Posted by b49P23TIvg
    This data line looks invalid, and my python installations (versions 2 and 3) agree:
    Code:
         "source":"<a rel="nofollow" href="http://twitter.com/">twitter< /a>",
    I understand that python raises an Exception. I'm afraid that figuring out why you were told "attribute not in module" rather than
    ValueError: Expecting , delimiter: line 1 column 268 (char 267)
    exceeds my paylevel.
    Hi ,Can you explain about pprint module

    Based on the below output I want to search for certain string

    ex: if search for 'tr' the output -> America\\/New_York only to display


    Code:
     'al': 'en-US,en;q=0.8',
     'c': 'US',
     'cy': 'Danvers',
     'g': 'A6qOVH',
     'gr': 'MA',
     'h': 'wfLQtf',
     'hc': 1331822918,
     'hh': '1.usa.gov',
     'l': 'orofrog',
     'll': [42.576698, -70.954903],
     'nk': 1,
     'r': 'http:\\//www.facebook.com\\/l\\/7AQEFzjSi\\/1.usa.gov\\/wfLQtf',
     't': 1331923247,
     'tz': 'America\\/New_York',
     'u': 'http:\\/\\/www.ncbi.nlm.nih.gov\\/pubmed\\/22415991'}
    '''
  14. #8
  15. Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Aug 2011
    Posts
    5,975
    Rep Power
    510
    pprint.pprint() pretty prints the object. pprint.pformat() returns the string that pprint.pprint() would print. There are some fine controls as well which I've almost never needed. pprint does not change the data. Here's a dictionary displayed with and without pprint.
    Code:
    >>> pprint.pprint({i:i for i in range(4)}, width=8)
    {0: 0,
     1: 1,
     2: 2,
     3: 3}
    >>> print({i:i for i in range(4)})
    {0: 0, 1: 1, 2: 2, 3: 3}
    >>>
    Next I recommend you spend a serious week trying experiments in the python interpreter as you carefully read the python tutorial.

    If d is a dictionary then d[key] is the value associated with the key. For example, you might want
    d['tz']

    In general to search a file I load it into emacs, where for 25 years I've reassigned C-s as isearch-forward-regexp . Then I use C-s (control s), type my regular expression and there it is. Found. Sometimes M-x occur followed by regular expression is useful. Without emacs I'm almost as inept as you. Without emacs neither gawk nor egrep are likely to be available as well.
    M-x delete-non-matching-lines is good to, depending on your editing goals.

    If you've only got python, then you can read the file, import re, and search the file. That's nasty work because of the trouble to write a program, think about syntax and printing. Don't write programs if you don't need to.
    [code]Code tags[/code] are essential for python code and Makefiles!
  16. #9
  17. Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Aug 2011
    Posts
    5,975
    Rep Power
    510
    duplicate post removed. I think my wifi works not right.

    Comments on this post

    • Will-O-The-Wisp agrees : It's happening to a few people around the Developer Shed Network - myself included. I'm looking into it! :)
    Last edited by b49P23TIvg; April 29th, 2015 at 02:27 PM. Reason: duplicate post removed
    [code]Code tags[/code] are essential for python code and Makefiles!

IMN logo majestic logo threadwatch logo seochat tools logo