#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2012
    Posts
    12
    Rep Power
    0

    Problem with urllib2


    Hi all,
    I am facing authentication errors while using urllib2. Pasted Below is the code i wrote, followed by the errors thrown. Could someone help me in this regard.

    Python version:
    D:\Data\Python Scripts\review>python
    ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
    Python 2.7.2 (default, Jun 24 2011, 12:22:14) [MSC v.1500 64 bit (AMD64)] on win32

    Code:
    password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()

    # Add the username and password. If we knew the realm, we could use it instead of ``None``.
    password_mgr.add_password(None, top_level_url, 'user', 'password')
    handler = urllib2.HTTPBasicAuthHandler(password_mgr)
    #--handler = urllib2.HTTPDigestAuthHandler(password_mgr)
    print "A: ", proxy_support
    # create "opener" (OpenerDirector instance)
    opener = urllib2.build_opener(handler)
    print "B: ", opener
    # Install the opener.
    # Now all calls to urllib2.urlopen use our opener.
    urllib2.install_opener(opener)
    # use the opener to fetch a URL
    obj = opener.open(top_level_url)



    Errors:


    D:\Data\Python Scripts\review>shpoint.py
    A: <urllib2.ProxyHandler instance at 0x00000000022D9448>
    B: <urllib2.OpenerDirector instance at 0x00000000022D91C8>
    Traceback (most recent call last):
    File "D:\Data\Python Scripts\review\shpoint.py", line 37, in <module>
    obj = getHTTPobject(mysite)
    File "D:\Data\Python Scripts\review\shpoint.py", line 30, in getHTTPobject
    obj = opener.open(top_level_url)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 859, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
    File "C:\ActivePython27\lib\urllib2.py", line 872, in retry_http_basic_auth
    return self.parent.open(req, timeout=req.timeout)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 859, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
    File "C:\ActivePython27\lib\urllib2.py", line 872, in retry_http_basic_auth
    return self.parent.open(req, timeout=req.timeout)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 859, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
    File "C:\ActivePython27\lib\urllib2.py", line 872, in retry_http_basic_auth
    return self.parent.open(req, timeout=req.timeout)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 859, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
    File "C:\ActivePython27\lib\urllib2.py", line 872, in retry_http_basic_auth
    return self.parent.open(req, timeout=req.timeout)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 859, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
    File "C:\ActivePython27\lib\urllib2.py", line 872, in retry_http_basic_auth
    return self.parent.open(req, timeout=req.timeout)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 859, in http_error_auth_reqed
    response = self.retry_http_basic_auth(host, req, realm)
    File "C:\ActivePython27\lib\urllib2.py", line 872, in retry_http_basic_auth
    return self.parent.open(req, timeout=req.timeout)
    File "C:\ActivePython27\lib\urllib2.py", line 400, in open
    response = meth(req, response)
    File "C:\ActivePython27\lib\urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
    File "C:\ActivePython27\lib\urllib2.py", line 432, in error
    result = self._call_chain(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 372, in _call_chain
    result = func(*args)
    File "C:\ActivePython27\lib\urllib2.py", line 884, in http_error_401
    url, req, headers)
    File "C:\ActivePython27\lib\urllib2.py", line 850, in http_error_auth_reqed
    headers, None)
    urllib2.HTTPError: HTTP Error 401: basic auth failed
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2012
    Posts
    39
    Rep Power
    3
    Do you know what type of authentication the server you are trying to access is using? Failing that what web server is it?
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2012
    Posts
    12
    Rep Power
    0
    I am trying to login to a Microsoft sharepoint server in our office, and I am not sure about the authentication method. Is there a way to find out in windows.

    Regards
    Sachin
  6. #4
  7. Banned ;)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Nov 2001
    Location
    Woodland Hills, Los Angeles County, California, USA
    Posts
    9,607
    Rep Power
    4247
    You should ask your sys administrator that question, after all he/she installed it. We use LDAP authentication for ours at work and I think the default authentication method is Windows Authentication. From your output of the urllib2 errors, looks like you're doing basic http authentication, which is also a completely different authentication protocol.
    Up the Irons
    What Would Jimi Do? Smash amps. Burn guitar. Take the groupies home.
    "Death Before Dishonour, my Friends!!" - Bruce D ickinson, Iron Maiden Aug 20, 2005 @ OzzFest
    Down with Sharon Osbourne

    "I wouldn't hire a butcher to fix my car. I also wouldn't hire a marketing firm to build my website." - Nilpo
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2012
    Posts
    39
    Rep Power
    3
    Microsoft normally use NTLM (NT LAN Manager) authentication although I don't have a Sharepoint server to confirm this.

    If you have pywin32 have a look in Lib\site-packages\win32\Demos\security\sspi. I think fetchurl would be the one to start with; it uses httplib rather than urllib2 but handles the authentication side.
  10. #6
  11. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2012
    Posts
    12
    Rep Power
    0
    Sorry for the late reply, but now I have something working. The authentication type used by Sharepoint as you correctly pointed out is "NTLM", so I traced a "python-ntlm" module (ver 1.0.1)

    and Here is what I did:
    1. installed python-ntlm module
    2. Added the following workaround in "HTTPNtlmAuthHandler.py" (found on code.google)
    auth_header_value = r.getheader(auth_header_field, None)
    + if ',' in auth_header_value:
    + auth_header_value, postfix = auth_header_value.split(',', 1)

    (ServerChallenge, NegotiateFlags) = ntlm.parse_NTLM_CHALLENGE_MESSAGE(auth_header_value[5:])
    3. used the WindowsHttpAuthenticated module, and the following code works.

    #-- Code Begins ---
    import urllib2
    from ntlm import HTTPNtlmAuthHandler

    user = 'domain-name\\user'
    password = "password"
    url = mysite

    passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
    passman.add_password(None, url, user, password)

    # create the NTLM authentication handler
    auth_NTLM = HTTPNtlmAuthHandler.HTTPNtlmAuthHandler(passman)

    # create and install the opener
    opener = urllib2.build_opener(auth_NTLM)
    urllib2.install_opener(opener)

    # retrieve the result
    response = urllib2.urlopen(url)
    print(response.read())
    #-- Code Ends ---

    Now, I don't see any errors and have some output. Next, I will try to use the SUDS module to work with Sharepoint. (As of now getting some "xml.sax._exceptions.SAXParseException" errors) But for that I will try a little more before posting a new question later.

    Warm Regards
    Sachin

IMN logo majestic logo threadwatch logo seochat tools logo