#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    4
    Rep Power
    0

    Function Question


    Greetings, I am trying to implement a function called webtext() that will take a URL and print the text of that website.
    Code:
    webtext()
  2. #2
  3. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2012
    Location
    39N 104.28W
    Posts
    158
    Rep Power
    3
    Take a look at the urllib2 module. Now, I know a lot of PyGurus say you should always use BeautifulSoup but I think it's too cumbersome, at least for me, at least for the needs I have.

    Here is a script I use to check David Thorne's blog to see if he's posted anything new:

    Code:
    import urllib2
    def countEnts():
        g=urllib2.urlopen('http://www.27bslash6.com')
        lstg=g.readlines()
        g.close()
        numentries=0
        srchstrng='<div id="bufferdotsindex">'
        for strl in lstg:
           if srchstrng in strl: numentries+=1
        return numentries
    print countEnts()
    As you can see, urllib2.urlopen() opens the website and treats it like a text file.

IMN logo majestic logo threadwatch logo seochat tools logo