#1
  1. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Posts
    6
    Rep Power
    0

    Apache freezing- help


    OK, I have two problems i was hoping to get some help with. The first is an emergency adn the second is just annoying.

    1st - I'm running apached on Red Hat 7.1. The OS never freezes, but the httpd service does about once a week at random intervals. a simple /etc/rc/d/init.d/httpd restart command gets her up again, but since I have a secure server with a key, requires a password from the console.

    A. How can I stop the freezing?

    B. Could I automate something to check and see if it was running, and then run that command WITH the pasword to restart it?

    2nd and much less importantly. From my intranet, I cannottype my fully qualified domain name (http://www.nationalpayments.com), i have to type http://web1 or http://10.0.0.3. Which is A, annoying, and B, gives me an error message when I restart.

    I do not think the two are related.

    Thanks so much in advance for any light the dev forum can shed =]

    Matt
  2. #2
  3. Full Access
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Jun 2000
    Location
    London, UK
    Posts
    2,019
    Rep Power
    20
    A. How can I stop the freezing?
    When you mean "freeze", does it simply stop responding to requests? I have had a similar problem, though much less frequently (twice in the year or so that the machine has been running - and only on one particular machine, not on any others). Unfortuately I wasn't even able to kill the Apache process - a reboot was required. Luckily this is only a development server.

    Next time it happens, run:
    Code:
    ps aux
    free
    and post the output here. Also post the last 20 lines of your error log:
    Code:
    tail -20 /var/log/httpd/error_log
    (or wherever your ErrorLog directive is set to). This should give us some useful diagnostic information.
    Alex
    (http://www.alex-greg.com)
  4. #3
  5. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Posts
    6
    Rep Power
    0
    alexgreg,

    Thanks for the respnse. Yes, freezing means apache stops respoding to page requests and "fails" to shut down during httpd restart, the is "ok" to start back up.

    During a normal restart of httpd, like I just ran and the results follow, both are "ok"

    [root@web1 /root]# /etc/rc.d/init.d/httpd restart
    Stopping httpd: [ OK ]
    Starting httpd: [Tue Jul 15 14:11:30 2003] [alert] httpd: Could not determine the server's fully qualified domain name, using 10.0.0.3 for ServerName
    Apache/1.3.27 mod_ssl/2.8.12 (Pass Phrase Dialog)

    Some of your private key files are encrypted for security reasons.In order to read them you have to provide us with the pass phrases.


    Server 10.0.0.3:443 (RSA)
    Enter pass phrase:

    Ok: Pass Phrase Dialog successful.


    Ok: Pass Phrase Dialog successful. [ OK ]


    The last 100+ error messages right now are comprised solely of these(and are all today by the way, which seems bad)

    bash: /root/.bashrc: Permission denied

    and variations of these:

    [Tue Jul 15 06:54:02 2003] [error] [client 204.17.108.246] File does not exist: /var/www/html/instmsg/aliases/operations

    Thanks,

    Matt
  6. #4
  7. Full Access
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Jun 2000
    Location
    London, UK
    Posts
    2,019
    Rep Power
    20
    How big is your log file? If it's within a reasonable size, can you please put it on a web server somewhere and post a link here? I'd like to check what's going on, sounds like something suspicious may be up. There's no reason why Apache should be trying to access root's bash configuration file...
    Alex
    (http://www.alex-greg.com)
  8. #5
  9. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Posts
    6
    Rep Power
    0
    Alex,

    I will absolutely post it when I get to work early am tomorrow. THANKS so much in advance. I'll post the link when it's live.

    Matt
  10. #6
  11. #7
  12. #8
  13. Full Access
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Jun 2000
    Location
    London, UK
    Posts
    2,019
    Rep Power
    20
    OK, something is attempting to run IIS exploits against your server. This is quite common, unfortunately - we have lots of them fired off against our servers here! Of course they're not vulnerable, since they're not running IIS.

    However, there is quite a serious issue: someone seems to have found a CGI script or something which can read arbitrary files on your server! Either that, or there is a badly programmed CGI attempting to access /root/.bashrc

    Any idea what might be doing this?

    You might want to create a blank file at /var/www/html/default.ida to stave off some of the 404 errors that are appearing.

    Next time the server crashes (and before you restart it) put the last 20 access_log entries and the last 20 error_log entries into separate files and post them here. Else, I'm just really speculating about what could be causing this.
    Alex
    (http://www.alex-greg.com)
  14. #9
  15. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Posts
    6
    Rep Power
    0
    Alex,

    Here's the info you wanted. Very interesting last error message...

    USER PID %CPU %MEM SIZE RSS TTY STAT START TIME COMMAND
    root 28087 0.0 0.1 1368 560 ? R 09:15 0:00 ps aux free PWD=/root
    [root@web1 /root]# ps aux
    USER PID %CPU %MEM SIZE RSS TTY STAT START TIME COMMAND
    apache 1231 0.0 0.0 1432 448 ? S Jul 17 0:00 ps daemon 820 0.0 0.1 1404 588 ? S Jul 17 0:00 (atd)
    mysql 936 0.0 1.0 27324 5224 ? S Jul 17 0:00 (mysqld)
    mysql 963 0.0 1.0 27324 5224 ? S Jul 17 0:00 (mysqld)
    mysql 964 0.0 1.0 27324 5224 ? S Jul 17 0:00 (mysqld)
    mysql 971 0.0 1.0 27324 5224 ? S Jul 17 0:00 (mysqld)
    postfix 1054 0.0 0.1 2268 972 ? S Jul 17 0:51 (qmgr)
    postfix 27854 0.0 0.1 2136 812 ? S 08:15 0:00 (pickup)
    postfix 28088 0.0 0.1 2200 884 ? S 09:15 0:00 (flush)
    root 1 0.0 0.1 1368 544 ? S Jul 17 0:04 init [5]
    root 2 0.0 0.0 0 0 ? SW Jul 17 0:00 (keventd)
    root 3 0.0 0.0 0 0 ? SW Jul 17 0:00 (kapmd)
    root 4 0.0 0.0 0 0 ? SWN Jul 17 0:00 (ksoftirqd_CPU0)
    root 5 0.0 0.0 0 0 ? SW Jul 17 0:03 (kswapd)
    root 6 0.0 0.0 0 0 ? SW Jul 17 0:01 (bdflush)
    root 7 0.0 0.0 0 0 ? SW Jul 17 0:00 (kupdated)
    root 8 0.0 0.0 0 0 ? SW Jul 17 0:00 (mdrecoveryd)
    root 76 0.0 0.0 0 0 ? SW Jul 17 0:00 (khubd)
    root 656 0.0 0.1 1432 640 ? S Jul 17 0:14 syslogd -m 0
    root 661 0.0 0.2 1972 1144 ? S Jul 17 0:00 klogd -2
    root 759 0.0 0.1 1352 532 ? S Jul 17 0:00 /usr/sbin/apmd -p 10
    root 808 0.0 0.1 1480 648 ? S Jul 17 0:00 /usr/sbin/automount -root 835 0.0 0.4 4780 2288 ? S Jul 17 0:00 /usr/sbin/snmpd -s -lroot 867 0.0 0.2 2568 1312 ? S Jul 17 0:13 xinetd -stayalive -reroot 894 0.0 0.2 2140 1032 ? S Jul 17 0:00 /bin/sh /usr/bin/saferoot 1050 0.0 0.1 2140 840 ? S Jul 17 0:01 (master)
    root 1073 0.0 0.0 1404 508 ? S Jul 17 0:00 gpm -t ps/2 -m /dev/mroot 1103 0.0 0.1 1552 700 ? S Jul 17 0:00 crond
    root 1159 0.0 0.2 4008 1468 ? S Jul 17 0:00 smbd -D
    root 1164 0.0 0.2 3264 1404 ? S Jul 17 0:01 nmbd -D
    root 1199 0.0 0.0 1340 436 1 S Jul 17 0:00 /sbin/mingetty tty1
    root 1200 0.0 0.0 1340 436 2 S Jul 17 0:00 /sbin/mingetty tty2
    root 1201 0.0 0.0 1340 436 3 S Jul 17 0:00 /sbin/mingetty tty3
    root 1202 0.0 0.0 1340 436 4 S Jul 17 0:00 /sbin/mingetty tty4
    root 1203 0.0 0.0 1340 436 5 S Jul 17 0:00 /sbin/mingetty tty5
    root 1204 0.0 0.0 1340 436 6 S Jul 17 0:00 /sbin/mingetty tty6
    root 1205 0.0 0.2 3324 1224 ? S Jul 17 0:00 (gdm)
    root 1212 0.0 1.7 19824 9252 ? S < Jul 17 0:03 (X)
    root 1213 0.0 0.3 3852 1620 ? S Jul 17 0:00 (gdm)
    root 1265 0.0 0.5 5172 3012 ? S Jul 17 0:02 (smbd)
    root 1284 0.0 0.7 7396 3968 ? S Jul 17 0:00 /usr/bin/gnome-sessioroot 1390 0.0 0.4 6144 2064 ? S Jul 17 0:00 gnome-smproxy --sm-coroot 1392 0.9 0.6 6996 3188 ? S Jul 17 66:11 magicdev --sm-config-root 1411 0.0 0.6 5520 3492 ? S Jul 17 0:00 sawfish --sm-client-iroot 1421 0.0 0.2 3180 1316 ? S Jul 17 0:00 gnome-name-service
    root 1429 0.0 1.2 9636 6624 ? S Jul 17 0:04 gmc --sm-config-prefiroot 1431 0.0 0.9 8536 5004 ? S Jul 17 0:00 panel --sm-config-preroot 1433 0.0 1.4 10596 7428 ? S Jul 17 0:00 gnome-help-browser --root 1438 0.0 0.7 7812 3928 ? S Jul 17 0:00 tasklist_applet --actroot 1440 0.0 0.7 7776 4064 ? S Jul 17 0:00 deskguide_applet --acroot 1534 0.0 0.4 5268 2192 ? S Jul 17 0:05 (smbd)
    root 2094 0.0 0.4 5364 2384 ? S Jul 17 0:10 (smbd)
    root 2461 0.0 0.4 5180 2128 ? S Jul 17 0:04 (smbd)
    root 2594 0.0 0.4 5352 2372 ? S Jul 17 0:27 (smbd)
    root 28008 0.0 0.1 1564 732 ? S 09:01 0:00 CROND
    root 28009 0.0 0.1 1920 904 ? S 09:01 0:00 /bin/bash /usr/bin/ruroot 28011 0.0 0.1 1656 552 ? S 09:01 0:00 awk -v progname=/etc/root 28012 0.0 0.1 1904 876 ? S 09:01 0:00 /bin/sh /usr/lib/sa/sroot 28014 0.0 0.1 1356 516 ? S 09:01 0:00 /usr/lib/sa/sadc 600
    root 28059 0.3 0.8 7412 4216 ? S 09:15 0:00 gnome-terminal --use-root 28061 0.0 0.1 1404 576 ? S 09:15 0:00 (gnome-pty-helpe)
    root 28062 0.0 0.2 2512 1460 ? S 09:15 0:00 bash
    root 28093 0.0 0.2 2508 1456 ? S 09:15 0:00 bash
    root 28118 0.8 1.0 8608 5300 ? S 09:15 0:00 emacs
    root 28123 0.0 0.1 1472 676 ? R 09:16 0:00 ps aux
    rpc 675 0.0 0.1 1512 588 ? S Jul 17 0:00 (portmap)
    xfs 1147 0.0 0.7 4980 3756 ? S Jul 17 0:00 (xfs)

    total used free shared buffers cached
    Mem: 514668 477164 37504 0 28968 376696
    -/+ buffers/cache: 71500 443168
    Swap: 530104 0 530104

    [root@web1 /root]# bash: tail -20 /var/log/httpd/error_log
    bash: bash:: command not found
    [root@web1 /root]# tail -20 /var/log/httpd/error_log
    [Mon Jul 21 13:06:03 2003] [error] [client 216.39.48.82] File does not exist: /var/www/html/robots.txt
    [Mon Jul 21 13:53:05 2003] [error] [client 10.0.0.10] File does not exist: /var/www/html/web
    [Mon Jul 21 14:04:40 2003] [error] [client 64.12.96.10] File does not exist: /var/www/html/kroger/,
    [Mon Jul 21 14:07:41 2003] [error] [client 64.12.96.10] File does not exist: /var/www/html/kroger/,
    [Mon Jul 21 14:08:44 2003] [error] [client 146.145.12.204] File does not exist: /var/www/html/favicon.ico
    [Mon Jul 21 14:21:54 2003] [error] [client 146.145.12.204] File does not exist: /var/www/html/favicon.ico
    [Mon Jul 21 14:59:45 2003] [error] [client 216.39.48.82] File does not exist: /var/www/html/robots.txt
    [Mon Jul 21 17:51:54 2003] [error] [client 10.0.0.18] File does not exist: /var/www/html/WEB
    [Mon Jul 21 18:13:46 2003] [error] [client 203.75.171.18] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /
    [Mon Jul 21 18:16:03 2003] [error] [client 216.39.48.82] File does not exist: /var/www/html/robots.txt
    [Mon Jul 21 19:36:07 2003] [error] [client 10.0.0.18] File does not exist: /var/www/html/WEB
    [Mon Jul 21 21:26:48 2003] [error] [client 216.39.48.82] File does not exist: /var/www/html/robots.txt
    [Mon Jul 21 22:09:09 2003] [error] [client 205.188.208.70] File does not exist: /var/www/html/downloads/matt/stang.html
    [Mon Jul 21 22:09:16 2003] [error] [client 205.188.208.39] File does not exist: /var/www/html/downloads/matt/stang.htm
    [Mon Jul 21 22:09:25 2003] [error] [client 205.188.209.82] File does not exist: /var/www/html/downloads/matt/
    [Mon Jul 21 22:09:52 2003] [error] [client 205.188.209.82] File does not exist: /var/www/html/downloads/matt/
    [Mon Jul 21 22:38:23 2003] [error] [client 10.0.0.12] File does not exist: /var/www/html/Web
    [Tue Jul 22 02:42:13 2003] [error] [client 216.39.48.82] File does not exist: /var/www/html/robots.txt
    [Tue Jul 22 04:02:01 2003] [notice] SIGHUP received. Attempting to restart
    [Tue Jul 22 04:02:02 2003] [alert] httpd: Could not determine the server's fully qualified domain name, using 10.0.0.3 for ServerName

IMN logo majestic logo threadwatch logo seochat tools logo