Ins and Outs
If you have kernel 2.6.20 or newer, with the CONFIG_TASK_IO_ACCOUNTING option built in, you can also use pidstat to monitor the I/O load for individual processes or whole groups. On a system with the right support, typing
pidstat -d 2 3
will generate output as shown in Figure 3. Pidstat will list the processes generating I/O and even tell you the type (read or write), along with the data throughput (in Kbps).
|
|
Figure 3: Linux administrators can now use pidstat to monitor I/O load for individual processes.
|
To allow this to happen, you need a recent kernel, the packages from the Sysstat website, and a matching cron configuration.
Listing 1 shows the /etc/cron.d/sysstat file on Ubuntu. In the simplest of all cases, the following two lines in /etc/crontab are all you need:
*/10 * * * * root /usr/lib/sa/sa1 1 1
53 23 * * * root /usr/lib/sa/sa2 -A
The first entry starts sadc (the System Activity Data Collector), the server daemon that collects data every second and stores its findings under /var/log/sa. The logfile format is saXX, where XX is the date. The second part with sar -A just rotates the logfiles.
01 # Global variables:
02 #
03 # our configuration file
04 DEFAULT=/etc/default/sysstat
05 # default setting, overriden in the above file
06 ENABLED=false
07 SA1_OPTIONS=""
08 # Activity reports every 10 minutes everyday
09 5-55/10 * * * * root [ -x /usr/lib/sysstat/sa1 ] && { [ -r "$DEFAULT" ] && . "$DEFAULT" ;
10 [ "$ENABLED" = "true" ] && exec /usr/lib/sysstat/sa1 $SA1_OPTIONS 1 1 ; }
11 # Additional run at 23:59 to rotate the statistics file
12 59 23 * * * root [ -x /usr/lib/sysstat/sa1 ] && { [ -r "$DEFAULT" ] && . "$DEFAULT" ; [
13 "$ENABLED" = "true" ] && exec /usr/lib/sysstat/sa1 $SA1_OPTIONS 60 2 ; }
Iostat and Mpstat
Besides sar and pidstat, the Sysstat package also includes iostat and mpstat. Mpstat monitors the processor you pass in to it; iostat is popular with file, database, and mail server administrators as a tool for measuring system I/O load by drive, disk, partition, and filesystem.
For example,
iostat -m -n 10 2
monitors an NFS mount point and outputs the volume of data in megabytes (Listing 2). For a single partition, you can do the same by typing iostat -m -x sda2 10 2, as shown in Listing 3.
01 # iostat -m -n 10 2
02 Filesystem: rMB_nor/s wMB_nor/s rMB_dir/s wMB_dir/s rMB_svr/s wMB_svr/s rops/s wops/s
03 server:/home 0,03 0,01 0,00 0,00 0,01 0,02 6,53 6,53
04
05 Filesystem: rMB_nor/s wMB_nor/s rMB_dir/s wMB_dir/s rMB_svr/s wMB_svr/s rops/s wops/s
06 server:/home 6,65 6,64 0,00 0,00 6,65 0,81 252,40 252,40
07 #
01 # iostat -m -x sda2 10 2
02 avg-cpu: %user %nice %system %iowait %steal %idle
03 2,01 0,01 0,52 0,40 0,00 97,05
04
05 Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
06 sda2 0,71 3,32 1,09 0,85 0,02 0,02 38,13 0,06 28,37 2,87 0,56
07
08 avg-cpu: %user %nice %system %iowait %steal %idle
09 6,14 0,00 0,81 0,33 0,00 92,71
10
11 Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
12 sda2 0,00 12,80 0,00 7,40 0,00 0,08 21,95 0,03 4,27 1,14 0,84
13 #
Absolutely Essential
The latest version of the Sysstat package includes some new features that many administrators have been waiting for – above all, the ability to monitor I/O load per process. Linux had some catching up to do in this regard. It is a pity you can't use this feature on older servers or on enterprise distributions. It is questionable whether distributions will respond with matching kernels and packages in future releases.
If you can do without detailed I/O statistics, a combination of sar and kSar will give you a flexible and comprehensive interface that lets you monitor multiple servers at the same time.
Even if you don't have a full-fledged monitoring system, you will still be able to identify recurring bottlenecks and take steps to fix any performance issues before a system crash.
Comments