Help with vmstat and apache processes

[RedCrayon]

Please help! I'm fairly new to server admin. I'm running an application on an Ubuntu (Jaunty) Linux box. Recently it's been getting very slow. I read that this could be due to problems with memory and/or I/O. I've run vmstat -S M 2 and have got this output (see attached image).

The problem is I've no idea how to interpret what's going on... The wa column seems really high and bi shoots up whenever someone searches the site. What's this telling me about my system??

Thanks

[coopstah13]

if you read the man page for vmstat, it says that the WA column under CPU is time spent waiting for IO, at 95% that means that 95% of your CPU time is being spent waiting for IO operations to complete, which is really bad, BI and BO are the blocks/s received and sent

[RedCrayon]

coopstah13 thanks for replying so quickly. I'd kind of got that far but I'm not sure what "waiting for IO operations to complete" means. i.e. What's causing there to be so much waiting time - (or the likely cause, or common causes)? What do I need to adjust in order to reduce the waiting time?

As I said I'm pretty new to all this so please forgive newbie questions...

[RedCrayon]

can anyone else help? I'm really rather stuck... thanks

[Irithori]

to get a more detailed view, install the sysstat package.

This command will reveal, which blockdevice is busy and with what:

iostat -mx 1

Look for %util. 100% is bad.
Then determine, what makes it busy. read, write or read and write?


After the device is clear, match it to the mountpoint.

cat /etc/fstab

With the mountpoint, look for applications accessing it

lsof |grep "<mountpoint>"

In general:
If you have multiple daemons using the same disk: bad
Even one daemon with multiple files open on the same disk (DB files, db temp files, logfiles, dumpfiles) is bad and could be splitted to multiple spindles.

Another point is to analyse your applications:
What are they reading/writing and how often?

etc, etc
But I hope you find a starting point here


Up to now, everything is possible:
- the app could be badly designed
- maybe simply the hardware limit of the disk is reached
- a degraded raid may also lead to reduced performance
- etc

[RedCrayon]

Thank you Irithori. I'll dig around and see what I can find! Much appreciated