#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2007
    Posts
    2
    Rep Power
    0

    Exclamation LDAP connection Pool | Weblogic threads getting stuck


    Hi,

    We are using Weblogic Portal 9.2 and external LDAP red hat directory server. We are using netscape api to perform LDAP operations.

    Recently we started using ConnectionPool class provided by netscape and after that we have started getting the exception shown below:

    =========================
    <Feb 7, 2007 4:25:40 PM IST> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '52' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "631" seconds working on the request "Http Request: /coltPortalWeb/coltPortal.portal", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
    Thread-793 "[STUCK] ExecuteThread: '52' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, waiting, priority=1, DAEMON> {
    -- Waiting for notification on: java.util.Vector@1cd188e[fat lock]
    jrockit.vm.Threads.waitForSignal(Native Method)
    java.lang.Object.wait(Native Method)
    netscape.ldap.util.ConnectionPool.getConnection(ConnectionPool.java:201)
    ^-- Lock released while waiting: java.util.Vector@1cd188e[fat lock]
    com.colt.util.LDAPUtil.<init>(LDAPUtil.java:254)
    com.colt.util.LDAPUtil.<init>(LDAPUtil.java:241)
    pageFlowControllers.loginFlow.LoginFlowController.LoginAction(LoginFlowController.java:201)
    jrockit.vm.RNI.c2java(Native Method)
    jrockit.reflect.NativeMethodInvoker.invoke0(Native Method)
    jrockit.reflect.NativeMethodInvoker.invoke(Unknown Source)
    jrockit.reflect.VirtualNativeMethodInvoker.invoke(Unknown Source)
    java.lang.reflect.Method.invoke(Unknown Source)
    org.apache.beehive.netui.pageflow.FlowController.invokeActionMethod(FlowController.java:869)
    org.apache.beehive.netui.pageflow.FlowController.getActionMethodForward(FlowController.java:808)
    org.apache.beehive.netui.pageflow.FlowController.internalExecute(FlowController.java:477)
    org.apache.beehive.netui.pageflow.PageFlowController.internalExecute(PageFlowController.java:305)
    org.apache.beehive.netui.pageflow.FlowController.execute(FlowController.java:335)
    ^-- Holding lock: pageFlowControllers.loginFlow.LoginFlowController@2bed6e0[thin lock]
    org.apache.beehive.netui.pageflow.internal.FlowControllerAction.execute(FlowControllerAction.java:51 )
    org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)
    org.apache.beehive.netui.pageflow.PageFlowRequestProcessor.access$201(PageFlowRequestProcessor.java: 95)
    org.apache.beehive.netui.pageflow.PageFlowRequestProcessor$ActionRunner.execute(PageFlowRequestProce ssor.java:2042)
    org.apache.beehive.netui.pageflow.interceptor.action.internal.ActionInterceptors$WrapActionIntercept orChain.continueChain(ActionInterceptors.java:63)
    org.apache.beehive.netui.pageflow.interceptor.action.ActionInterceptor.wrapAction(ActionInterceptor. java:167)
    =========================

    Can somebody advise abt how to resolve this? Any help will be greatly appreciated.

    Pls let me know if more details are required.

    regards,
    Ankur
  2. #2
  3. Modding: Oracle MsSQL Firebird
    Devshed Supreme Being (6500+ posts)

    Join Date
    Jun 2001
    Location
    Outside US
    Posts
    8,527
    Rep Power
    538
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2007
    Posts
    2
    Rep Power
    0
    Not sure if it is a bug in netscape API. Have you heard of any bug like that?

    Yes we are using LDAP v3.

    Since we are already end of development switching to a new API might not be a feasible option :-(
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2007
    Posts
    0
    Rep Power
    0
    Hi,
    Looks like we are having the same issue with Weblogic 8.1 LDAP access. Sometimes in the production environement it is just frozen and all the threads are stuck. We tried to use some timeout for LDAP connection but the time out value is ignored. Have you found any solution for the problem?
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2009
    Posts
    1
    Rep Power
    0
    Hi,

    I am getting similar error - any help would be appreciated. THX


    Thread-1533 "[STUCK] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, suspended, waiting, priority=1, DAEMON> {
    -- Waiting for notification on: netscape.ldap.LDAPResponseListener@ffffffff80fe87c1[fat lock]
    java.lang.Object.wait(Object.java:474)
    netscape.ldap.LDAPMessageQueue.waitForMessage(Unknown Source)
    netscape.ldap.LDAPMessageQueue.waitFirstMessage(Unknown Source)
    netscape.ldap.LDAPConnection.sendRequest(Unknown Source)
    ^-- Holding lock: netscape.ldap.LDAPConnection@ffffffff80fe7aad[thin lock]
    netscape.ldap.LDAPConnection.simpleBind(Unknown Source)
    netscape.ldap.LDAPConnection.authenticate(Unknown Source)
    netscape.ldap.LDAPConnection.authenticate(Unknown Source)
    netscape.ldap.LDAPConnection.bind(Unknown Source)
    weblogic.security.providers.authentication.LDAPAtnDelegate$LDAPFactory.newInstance(LDAPAtnDelegate.j ava:3687)
    weblogic.security.utils.Pool.newInstance(Pool.java:37)
    weblogic.security.utils.Pool.getInstance(Pool.java:29)
    weblogic.security.providers.authentication.LDAPAtnDelegate.getConnection(LDAPAtnDelegate.java:3142)
    weblogic.security.providers.authentication.LDAPAtnDelegate.userExists(LDAPAtnDelegate.java:2260)
    weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java: 136)
    com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
    com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:101)
    sun.reflect.GeneratedMethodAccessor9101.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:570)
    javax.security.auth.login.LoginContext.invoke(LoginContext.java:720)
    javax.security.auth.login.LoginContext.access$000(LoginContext.java:186)
    javax.security.auth.login.LoginContext$4.run(LoginContext.java:683)
    javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    javax.security.auth.login.LoginContext.login(LoginContext.java:566)
    com.bea.common.security.internal.service.JAASLoginServiceImpl.login(JAASLoginServiceImpl.java:71)
    sun.reflect.GeneratedMethodAccessor1066.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:570)
    com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
    $Proxy11.login(Unknown Source)
    weblogic.security.service.internal.WLSJAASLoginServiceImpl$ServiceImpl.login(Unknown Source)
    com.bea.common.security.internal.service.IdentityAssertionCallbackServiceImpl.assertIdentity(Identit yAssertionCallbackServiceImpl.java:96)
    sun.reflect.GeneratedMethodAccessor702.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:570)
    com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
    $Proxy14.assertIdentity(Unknown Source)
    com.bea.common.security.internal.service.IdentityAssertionServiceImpl.assertIdentity(IdentityAsserti onServiceImpl.java:78)
    sun.reflect.GeneratedMethodAccessor699.invoke(Unknown Source)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    java.lang.reflect.Method.invoke(Method.java:570)
    com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
    $Proxy35.assertIdentity(Unknown Source)
    weblogic.security.service.PrincipalAuthenticator.assertIdentity(Unknown Source)
    weblogic.servlet.security.internal.CertSecurityModule.assertIdentity(CertSecurityModule.java:103)
    weblogic.servlet.security.internal.CertSecurityModule.checkUserPerm(CertSecurityModule.java:65)
    weblogic.servlet.security.internal.SecurityModule.checkAccess(SecurityModule.java:107)
    weblogic.servlet.security.internal.ServletSecurityManager.checkAccess(ServletSecurityManager.java:57 )
    weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2076)
    weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1998)
    weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1331)
    weblogic.work.ExecuteThread.execute(ExecuteThread.java:197)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:164)
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2009
    Location
    Charlotte, NC
    Posts
    111
    Rep Power
    9
    I suggest changing the function your using from getConnection() to getConnFromPool() or increasing your connection pool. The exception reports a (fat lock) which indicates there were no available connections in the pool, so the API just waited and eventually timed out. By using getConnFromPool() it will return a NULL if all the connection in the pool are busy and not hang the JVM. Also make sure your code is calling close() after each login, if not then this may be the cause of the pool running dry. I have seen where failed logins are not handled properly and a close() not called in the TryCatch statement and after enough failed logins the JVM hangs. Good Luck!
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Nov 2012
    Posts
    1
    Rep Power
    0

    LDAP connection issue after restarting the LDAP server


    Please find the thread dumps when the LDAP connection issue occurs:

    "pool-94-thread-51" prio=10 tid=0x00002aaba5a57800 nid=0x6691 in Object.wait() [0x0000000065cb4000..0x0000000065cb4b10]
    java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00002aaaf3f0f190> (a netscape.ldap.LDAPResponseListener)
    at java.lang.Object.wait(Object.java:502)
    at netscape.ldap.LDAPMessageQueue.waitForMessage(Unknown Source)
    - locked <0x00002aaaf3f0f190> (a netscape.ldap.LDAPResponseListener)
    at netscape.ldap.LDAPMessageQueue.waitFirstMessage(Unknown Source)
    - locked <0x00002aaaf3f0f190> (a netscape.ldap.LDAPResponseListener)
    at netscape.ldap.LDAPConnection.sendRequest(Unknown Source)
    - locked <0x00002aaad998bf48> (a netscape.ldap.LDAPConnection)
    at netscape.ldap.LDAPConnection.simpleBind(Unknown Source)
    at netscape.ldap.LDAPConnection.restoreConnection(Unknown Source)
    at netscape.ldap.LDAPConnection.checkConnection(Unknown Source)
    at netscape.ldap.LDAPConnection.search(Unknown Source)
    at netscape.ldap.LDAPConnection.read(Unknown Source)
    at netscape.ldap.LDAPConnection.read(Unknown Souurce)


    Please help to resolve this,this is very critical issue.

IMN logo majestic logo threadwatch logo seochat tools logo