Linux Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed Forums > Operating Systems > Linux Help
Reload this Page

RHEL 5 - LAMP - High Load issue 10.0+

Discuss RHEL 5 - LAMP - High Load issue 10.0+ in the Linux Help forum on Dev Shed.
RHEL 5 - LAMP - High Load issue 10.0+ Linux Help forum discussing topics including usage, troubleshooting, modules, and distributions. Linux is an open source OS, based on UNIX.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
Stop making mediocre tutorials.The best tutorials are video! Camtasia Studio makes it easy to create engaging, buzz-building screen videos at any size, in any popular format. Download the free trial!
  #1  
Old April 29th, 2008, 02:25 PM
MPI MPI is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Mar 2008
Posts: 25 MPI User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 h 16 m 26 sec
Reputation Power: 0
RHEL 5 - LAMP - High Load issue 10.0+
i have a LMAP platform. mysql has it's own dedicated box.

both the apache and the mysql server have dual 3.0 xeons, with 8 to 12 gigs of ram. yes, overkill

tho the apache server has cpanel installed on it, both servers serve just 1 website/domain.

main php values:
Code:
max_execution_time = 10     ; Maximum execution time of each script, in seconds
max_input_time = 10     ; Maximum amount of time each script may spend parsing request data
memory_limit = 16M      ; Maximum amount of memory a script may consume (32MB)
eaccelerator.shm_size = "16"


btw, i installed php e_accelerator 3 days ago but it hasn't helped at all.

i thought the load problem was mysql, optimized it to the max, the load issue is still there, then i thought it was apache. but apache status shows a very light cpu usage:

Code:
Apache:
0.907% CPU load

Top:
load average: 11.01, 9.97, 14.81
Cpu(s):  0.9%us,  1.1%sy,  0.0%ni, 63.3%id, 33.3%wa,  0.1%hi,  1.1%si,  0.2%st

Free: (in MEGS)
             total       used       free     shared    buffers     cached
Mem:         12056      11989         66          0         36      10818
-/+ buffers/cache:       1134      10921
Swap:        23999         19      23980


so now i'm gonna take a look at other services that are running, when i installed the OS on the server, i installed ALOT of stuff that i didn't need. ...so i'm thinking maybe one of those apps is causing it, but after going thru the list of:

service --status-all > /services.txt

...i don't recognize half the services, and i'm scared to stop them because i'm not sure what they do.

Code:
acpid (pid 4727) is running...
amd is stopped
anacron is stopped
antirelayd (pid 4956) is running...
arpwatch is stopped
atd is stopped
auditd (pid 4293) is running...
automount (pid 4703) is running...
Avahi daemon is running
Avahi DNS daemon is not running
Usage: bandmin {start|stop}
hcid (pid 4561) is running...
sdpd (pid 4567) is running...
chkservd (pid 11864) is running...
conmand is stopped
cpsrvd (pid  12057) is running...
entropychat (pid 5201) is running...
melange is stopped
stunnel is stopped
interchange is stopped
qrunner is stopped
Frequency scaling not supported under xen kernels
crond (pid 5064) is running...
cupsd (pid 4765) is running...
cups-config-daemon is obsolete
dsm_sa_datamgr32d (pid 7989 7701) is running
dsm_sa_eventmgr32d (pid 7735) is running
dsm_sa_snmp32d (pid 7745) is running
dc_client is stopped
dc_server is stopped
dnsmasq (pid 7931) is running...
dsm_om_connsvc32d (pid 7781 7780) is running
dsm_om_shrsvc32d (pid 7262) is running
ipmi_msghandler module loaded.
ipmi_si module loaded.
ipmi_devintf module loaded.
/dev/ipmi0 exists.
dund is stopped
exim (pid 4947 4944) is running...
Usage: filelimits {start|stop|restart|reload}
Usage: /etc/init.d/firstboot {start|stop}
gpm is stopped
hald (pid 7974) is running...
hidd (pid 4676) is running...
hpiod (pid 4743) is running...
hpssd (pid 4748) is running...

               Apache Server Status for host.domain.com

   Server Version: Apache/1.3.37 (Unix) PHP/4.4.7 mod_ssl/2.8.28 OpenSSL/0.9.8b
   Server Built: Aug 13 2007 01:45:04
     _________________________________________________________________

   Current Time: Tuesday, 29-Apr-2008 12:14:35 PDT
   Restart Time: Tuesday, 29-Apr-2008 11:52:32 PDT
   Parent Server Generation: 0
   Server uptime: 22 minutes 3 seconds
   Total accesses: 42196 - Total Traffic: 86.4 MB
   CPU Usage: u4.26 s9.58 cu.26 cs.05 - 1.07% CPU load
   31.9 requests/sec - 66.9 kB/second - 2147 B/request
   1239 requests currently being processed, 25 idle servers
WWW_WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWKWWWWWWWWWWWW
WWWWWWWW_WWWWWWWWWWWRWWKWWWWWWWWWWKWWWWWWWWWWWWWWWWWWKWWWWWWWWWW
WWWWWWWWWWWWWWKWWWKWWWKWWWWKWWWWWWWWWWWWWWWWWWWWWWWWWWWWWKWWKWKW
WWWWWKWWKWWWWWWWWWWWWWWWWWWWWWKWWWWKWWWWWWWWWWWWWWWWWWWWWWWWWWW.
WWWWKWWWWWWKWWW.WWWWWWWWWWWWWWWW.WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWRWWWWWWWWKKWWWWWWWWWWWWWWWWWKWWKWWWKWWWWWWWWWWRWWRWWWKWWWWWW
WWWWWWK_WWWWWWWWW_WWW.WKWWWWWWWWWWWWWWWWWWWWWWWWWWW_WWW_WKWWWWRW
WWWWWWWWWWKKWWWWWWWWWWWWWWWWWWWWWWKWWWWWKWWWKWWWWWWWWWWWWWWWKWWW
WWW_WWWWWWWWWWWWWW_WWWWWWWWWWWWWWWWKW.WK.WWWWWWWW_WWWWWWRWWWKWW_
WWWKWWWWWWWWWKWWWWWWWWWWWWWWWWWWKKWWWWWWWWWKWWWWWWWKWKWW_WWWWWWW
WWWWWWWW.WWWWWWWWWWWWWWWWKWWWWWWWWWWWWWWWKWWWKWWWRWWWWWWWWWWWWWW
WWWKWWWWKWRWW_WWWWWKWKWKWWWWKWKWWWWWRWWWWWWWWWWWWWWWWRWWWWWWKKWK
KWWWWWWWWWWWWWWWWWWWWWWKWWWWWWWWWWKWW_KWWKWWRWWWWWWWWWWWWKWWWWWW
KWWWWKWWWWKWWWWWWWKWWWWWWW_WWWWWWWWWWWWKRWWWWWWWWWWWWWWWWWWWWWWW
WWWKWWWWWWWKWWWWWWWWKWWWWWWWWKWWWWWKWWWWWWWWWWW_WWWKWWWKWWKWWWWW
K_WWWWWWW.WWWWWWKWKWKWWWW_WWKWWWWWWWWWWRWWWWWWWWWWWWWWWWWKWWWWKW
KKWWWWWWWKWWKKWWWWWKWWWWKWWWWWWWWWWWWWWWWWWWKKWWWKWWWWKWWWWWWWWW
WKWWWWWWWKWWWWWWWWWKKWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWK.WWWWWKWWWWKWWRWWWWWWWWWWWKWWWWWWWW_WWWWWWWKKRWWKKWWKWWKWW
WW_WWWWWW__WWWW_WWWKWKKW_K_WKWWKWWW_WRWKKKKWKKWKWKWKKKKK..W.....
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
........................................................

   Scoreboard Key:
   "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
   "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
   "L" Logging, "G" Gracefully finishing, "." Open slot with no current process
ibmsphalt is stopped
dell_rbu (module) is running
ipmi driver is running
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    RH-Firewall-1-INPUT  all      ::/0                 ::/0               

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    RH-Firewall-1-INPUT  all      ::/0                 ::/0               

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Chain RH-Firewall-1-INPUT (2 references)
num  target     prot opt source               destination         
1    ACCEPT     all      ::/0                 ::/0               
2    ACCEPT     icmpv6    ::/0                 ::/0               
3    ACCEPT     esp      ::/0                 ::/0               
4    ACCEPT     ah       ::/0                 ::/0               
5    ACCEPT     udp      ::/0                 ff02::fb/128       udp dpt:5353 
6    ACCEPT     udp      ::/0                 ::/0               udp dpt:631 
7    ACCEPT     tcp      ::/0                 ::/0               tcp dpt:631 
8    ACCEPT     udp      ::/0                 ::/0               udp dpts:32768:61000 
9    ACCEPT     tcp      ::/0                 ::/0               tcp dpts:32768:61000 
10   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:1311 
11   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:2083 
12   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:2087 
13   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:2096 
14   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:22 
15   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:25 
16   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:80 
17   ACCEPT     tcp      ::/0                 ::/0               tcp dpt:443 
18   REJECT     all      ::/0                 ::/0               reject-with icmp6-port-unreachable 

Usage: ipaliases {start|stop|restart|reload}
ipmi_msghandler module loaded.
ipmi_si module loaded.
ipmi_devintf module loaded.
/dev/ipmi0 exists.
Table: nat
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination         
1    MASQUERADE  all  --  192.168.122.0/24     0.0.0.0/0           

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         

Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:53 
2    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:53 
3    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:67 
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:67 
5    acctboth   all  --  0.0.0.0/0            0.0.0.0/0           
6    RH-Firewall-1-INPUT  all  --  0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED 
2    ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
5    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
6    RH-Firewall-1-INPUT  all  --  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    acctboth   all  --  0.0.0.0/0            0.0.0.0/0           

Chain RH-Firewall-1-INPUT (2 references)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           icmp type 255 
3    ACCEPT     esp  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     ah   --  0.0.0.0/0            0.0.0.0/0           
5    ACCEPT     udp  --  0.0.0.0/0            224.0.0.251         udp dpt:5353 
6    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:631 
7    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:631 
8    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
9    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:1311 
10   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:2083 
11   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:2087 
12   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:2096 
13   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
14   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:25 
15   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:80 
16   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:443 
17   REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain acctboth (2 references)
num  target     prot opt source               destination         
1               tcp  --  192.168.123.6        0.0.0.0/0           tcp dpt:80 
2               tcp  --  0.0.0.0/0            192.168.123.6       tcp spt:80 
3               tcp  --  192.168.123.6        0.0.0.0/0           tcp dpt:25 
4               tcp  --  0.0.0.0/0            192.168.123.6       tcp spt:25 
5               tcp  --  192.168.123.6        0.0.0.0/0           tcp dpt:110 
6               tcp  --  0.0.0.0/0            192.168.123.6       tcp spt:110 
7               icmp --  192.168.123.6        0.0.0.0/0           
8               icmp --  0.0.0.0/0            192.168.123.6       
9               tcp  --  192.168.123.6        0.0.0.0/0           
10              tcp  --  0.0.0.0/0            192.168.123.6       
11              udp  --  192.168.123.6        0.0.0.0/0           
12              udp  --  0.0.0.0/0            192.168.123.6       
13              all  --  192.168.123.6        0.0.0.0/0           
14              all  --  0.0.0.0/0            192.168.123.6       
15              tcp  --  192.168.122.1        0.0.0.0/0           tcp dpt:80 
16              tcp  --  0.0.0.0/0            192.168.122.1       tcp spt:80 
17              tcp  --  192.168.122.1        0.0.0.0/0           tcp dpt:25 
18              tcp  --  0.0.0.0/0            192.168.122.1       tcp spt:25 
19              tcp  --  192.168.122.1        0.0.0.0/0           tcp dpt:110 
20              tcp  --  0.0.0.0/0            192.168.122.1       tcp spt:110 
21              icmp --  192.168.122.1        0.0.0.0/0           
22              icmp --  0.0.0.0/0            192.168.122.1       
23              tcp  --  192.168.122.1        0.0.0.0/0           
24              tcp  --  0.0.0.0/0            192.168.122.1       
25              udp  --  192.168.122.1        0.0.0.0/0           
26              udp  --  0.0.0.0/0            192.168.122.1       
27              all  --  192.168.122.1        0.0.0.0/0           
28              all  --  0.0.0.0/0            192.168.122.1       
29              all  --  0.0.0.0/0            0.0.0.0/0           

irattach is stopped
irqbalance (pid 4363) is running...
iscsid (pid 3608 3607) is running...
iscsid (pid 3608 3607) is running...
Kdump is not operational
krb524d is stopped
libvirt_qemud (pid 7816) is running...
lisa is stopped
mdmpd is stopped
dbus-daemon (pid 4545) is running...
Usage /etc/init.d/mptctl {start|stop}
multipathd is stopped
Usage: /etc/init.d/mysql start|stop|restart|reload
number of zones: 10
debug level: 0
xfers running: 0
xfers deferred: 0
soa queries in progress: 0
query logging is OFF
recursive clients: 0/1000
tcp clients: 0/100
server is up and running
netconsole module not loaded
netplugd is stopped
Configured devices:
lo eth0 eth1
Currently active devices:
lo peth0 virbr0 vif0.0 eth0 xenbr0
NetworkManager is stopped
NetworkManagerDispatcher is stopped
rpc.mountd is stopped
nfsd is stopped
rpc.rquotad is stopped
rpc.statd dead but pid file exists
nscd is stopped
ntpd is stopped
Low level hardware support loaded: none found


The following modules are also loaded:
	rdma_cm
	ib_cm
	ib_iser

pand is stopped
pcscd (pid 4648) is running...
pkcsslotd is stopped
portmap is stopped
portsentry (pid 8051) is running...
Process accounting is disabled.
pure-ftpd (pid 5044) is running...

+------+---------+-------+------+-------------------------------------------+
| PID  |  Login  |For/Spd| What |                 File/IP                   |
+------+---------+-------+------+-------------------------------------------+

Checking Remote Access Controller (RAC4) Service... not running [  OK  ]
rdisc is stopped
rhnsd (pid 7844) is running...
rpc.idmapd (pid 4494) is running...
saslauthd is stopped
init (pid 1) is running...
smartd (pid 8751) is running...
snmpd is stopped
snmptrapd is stopped
squid is stopped
sshd (pid 29786 28366 4784) is running...
syslogd (pid 4343) is running...
klogd (pid 4346) is running...
CIM server is not running
tux is stopped
Xvnc is stopped
winbindd is stopped
wpa_supplicant is stopped
xend is running
Checking for xendomains:[unused][FAILED]
xfs is stopped
xinetd (pid 4801) is running...
ypbind is stopped
yum-updatesd (pid 7940) is running...


i'd appreciate it if anyone can go thru the list i posted above and they see anything unsual running.
Reply With Quote
  #2  
Old April 30th, 2008, 02:51 AM
SimonJM SimonJM is offline
Contributing User
Dev Shed Novice (500 - 999 posts) Click here for more information
 
Join Date: Mar 2006
Posts: 609 SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 1 Week 5 Days 8 m 27 sec
Reputation Power: 194
That's a load average higher than I would expect/like, especially with such a low cpu utilisation (user and system total 2%). Judging from the rest of your info I'd start looking at disk I/O as you seem to be 33% wait on I/O and that can cause things to stack up.
__________________
"Do not meddle in the affairs of cats, for they are subtle and will pee on your computer" - Bruce Graham
Reply With Quote
  #3  
Old April 30th, 2008, 01:31 PM
MPI MPI is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Mar 2008
Posts: 25 MPI User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 h 16 m 26 sec
Reputation Power: 0
Quote:
Originally Posted by SimonJM
That's a load average higher than I would expect/like, especially with such a low cpu utilisation (user and system total 2%). Judging from the rest of your info I'd start looking at disk I/O as you seem to be 33% wait on I/O and that can cause things to stack up.


i bet that's the problem right there, thx!

from what i've read so far, iowait can be either network or disk, assuming that i'm correct, all my network cards are onboard GigE and i'm not pushing anywhere near 1000megs/sec, but tho my mysql and apache servers disks are all fast SCSIs, my DAS (storage server) has only SATA2s.

can someone please tell me how to find out where the iowait is coming from?

some more stats
Code:
[root@host ~]# mpstat 5
Linux 2.6.18-53.1.4.el5xen   04/30/2008

11:21:36 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
11:21:41 AM  all    0.70    0.00    1.35   28.34    0.05    2.00    0.25   67.32   7068.86
11:21:46 AM  all    0.50    0.00    1.05   28.99    0.05    1.95    0.20   67.27   7055.00
11:21:51 AM  all    0.50    0.00    1.40   42.59    0.10    2.00    0.25   53.17   7515.80
11:21:56 AM  all    0.85    0.00    1.75   31.61    0.00    2.20    0.20   63.38   7271.06
11:22:01 AM  all    0.60    0.00    1.95   27.50    0.10    2.15    0.20   67.51   7310.20
11:22:06 AM  all    1.35    0.00    2.35   26.36    0.10    2.10    0.15   67.60   7377.20
11:22:11 AM  all    0.75    0.00    1.75   30.13    0.10    2.20    0.20   64.87   7567.27
11:22:16 AM  all    0.75    0.00    1.40   26.66    0.05    1.75    0.20   69.18   7567.13
11:22:21 AM  all    1.20    0.00    1.90   29.93    0.05    1.85    0.15   64.91   7473.20
11:22:26 AM  all    0.85    0.00    1.75   28.66    0.05    2.30    0.25   66.13   7499.00
11:22:31 AM  all    0.75    0.00    2.20   26.84    0.05    2.35    0.15   67.67   7395.41
11:22:36 AM  all    0.60    0.00    1.40   27.12    0.05    1.85    0.20   68.78   7388.42
11:22:41 AM  all    0.70    0.00    1.45   27.71    0.15    2.05    0.20   67.73   7403.01
11:22:46 AM  all    0.60    0.00    1.55   27.20    0.10    1.95    0.20   68.40   7599.60
11:22:51 AM  all    0.45    0.00    1.40   26.10    0.10    1.95    0.20   69.80   7441.20
11:22:56 AM  all    0.45    0.00    1.50   28.69    0.15    2.05    0.20   66.97   7516.37
11:23:01 AM  all    1.00    0.00    1.60   26.30    0.20    2.10    0.15   68.65   7628.06
11:23:06 AM  all    1.65    0.00    3.10   26.85    0.05    2.15    0.15   66.05   7719.80
11:23:11 AM  all    0.70    0.00    1.35   30.85    0.10    1.75    0.25   65.00   7683.07
11:23:16 AM  all    0.60    0.00    1.20   26.16    0.05    2.10    0.25   69.63   7691.16
11:23:21 AM  all    0.70    0.00    1.45   24.91    0.15    1.65    0.20   70.94   7620.60
11:23:26 AM  all    8.46    0.00    2.00   28.18    0.15    2.60    0.20   58.41   7589.20
11:23:31 AM  all    0.95    0.00    1.55   28.01    0.10    1.85    0.20   67.33   7476.05
11:23:36 AM  all    0.65    0.00    1.40   26.21    0.15    2.30    0.20   69.10   7550.80

[root@host ~]# sar -u 5 5
Linux 2.6.18-53.1.4.el5xen   04/30/2008

11:24:15 AM       CPU     %user     %nice   %system   %iowait    %steal     %idle
11:24:20 AM       all      0.85      0.00      5.05     31.78      0.35     61.97
11:24:25 AM       all      0.60      0.00      4.51     31.35      0.15     63.40
11:24:30 AM       all      1.15      0.00      4.40     28.94      0.20     65.32
11:24:35 AM       all      1.10      0.00      4.30     28.25      0.20     66.15
11:24:40 AM       all      0.65      0.00      4.00     29.58      0.15     65.62
Average:          all      0.87      0.00      4.45     29.98      0.21     64.49


Code:
[root@host ~]# iostat
Linux 2.6.18-53.1.4.el5xen   04/30/2008

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.92    0.00    2.28   33.32    0.19   63.30

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               3.70        45.13        50.53   42574972   47674346
dm-0              8.42        44.99        50.34   42445194   47494192
dm-1              0.04         0.13         0.19     127136     180144
sdb              11.64      1384.00         3.94 1305724925    3721648
sdc               5.02       575.91         1.74  543335032    1642992
sdd              14.27      1637.71         6.15 1545083914    5804936
sde             109.08      2775.92       219.45 2618918714  207035936
sdf               0.00         0.00         0.00        644          0


...ps...device SDE is a few 500gb sata2 7,200k rpm disks, for now, i switched over a few of the files over to device SDC, which contains three 300gb 10k sas disks configured as a raid 5.

if i switch everything over to sas disks, will that fix the problem?

will changing any of the following affect the iowait:
read policy
write policy
Stripe Element Size

..?
Reply With Quote
  #4  
Old May 1st, 2008, 03:55 PM
MPI MPI is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Mar 2008
Posts: 25 MPI User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 h 16 m 26 sec
Reputation Power: 0
anyone..?
Reply With Quote
  #5  
Old May 2nd, 2008, 07:02 PM
MPI MPI is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
  
Join Date: Mar 2008
Posts: 25 MPI User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 h 16 m 26 sec
Reputation Power: 0
bump
Reply With Quote
  #6  
Old May 6th, 2008, 03:58 AM
SimonJM SimonJM is offline
Contributing User
Dev Shed Novice (500 - 999 posts) Click here for more information
 
Join Date: Mar 2006
Posts: 609 SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level)SimonJM User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 1 Week 5 Days 8 m 27 sec
Reputation Power: 194
Been on holiday!

Try sar -d to get disk stats ... man sar for how to invoke. Failing that use sa1 from cron to get daily output files in /var/adm/sa call sadd - the dd will be day number, which you can later interrogate with sar -f filename.
Reply With Quote
Reply

Viewing: Dev Shed Forums > Operating Systems > Linux Help > RHEL 5 - LAMP - High Load issue 10.0+

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

 Free IT White Papers!
 
Accelerating Trading Partner Performance
One in five. That's how many partner transactions have at least one error. That is an amazing statistic, particularly given the extraordinary leaps in innovation across the global supply chain during the past two decades. Download this white paper to learn more.
 
Competing on Analytics
This Tech Analysis is designed to help identify characteristics shared by analytics competitors, and includes information about 32 organizations that have made a commitment to quantitative, fact-based analysis.
 
Cost Effective Scaling with Virtualization and Coyote Point Systems
An overview of the industry trend toward virtualization, how server consolidation has increased the importance of application uptime and the steps being taken to integrate load balancing technology with virtualized servers.
 
Five Checkpoints to Implementing IP Telephony
Implementation planning for IP PBX software and IP telephony has become vital as businesses replace discontinued legacy PBX phone systems. This informative whitepaper outlines five "checkpoints" for any implementation plan that will help make IP communications a successful proposition.
 
Hosted Email Security: Staying Ahead of New Threats
In the last two years, email has become a fierce battleground between the nefarious forces of spam and malware, and the heroes of messaging protection. The spam volumes increased alarmingly every month, bringing clever new forms of phishing and virus propagation attacks.
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 1 hosted by Hostway.