26 June 2007

Script to alert on high disk utilization

Below I've got a simple script that will check your disk utilization every time it's run and then alert you if it exceeds a certain level. It could be run as often as you like from cron. This script is not pretty but it's effective. It should probably be modified for your particular situation before you use it.

Although it's pretty simple, some parts of it may be mysterious if you're not familiar with the various Unix utilities it uses. Let's take a look at the code first, then I'll briefly explain some of the parts which you may want to tweak, and I'll touch on some of the other parts that would be useful to play with on the command line to learn more. So, to the script:

#!/usr/bin/env bash

if ! df -t ufs |grep ' \(9[5-9]\|10[0-9]\)% ' > /dev/null ; then
   exit 0

for MP in `df -t ufs | grep ' 9[5-9]% ' | awk '{ print $6 }'` ; do
   PU=`df -t ufs | grep " ${MP}$" | awk '{ print $5 }'`
   MSG="file system $MP at $PU"
   logger -p local3.warn -t `hostname | sed 's/\..*$//'` $MSG

for MP in `df -t ufs | grep ' 10[0-9]% ' | awk '{ print $6 }'` ; do
   PU=`df -t ufs | grep " ${MP}$" | awk '{ print $5 }'`
   MSG="${MP}: file system full, $PU"
   logger -p local3.crit -t `hostname | sed 's/\..*$//'` $MSG

What this script is doing is just checking the output of the df (disk free) command and then sending a message to the system log if it exceeds a certain level. I've got separate loops for high disk utilization and full disk utilization (100%+) so that you can use different messages. The first, small code block, the if clause, is to improve efficiency by running df just once when there is no alerting condition (presumably, the case most of the time).

One thing you might want to change in this script is the logging commands, which use logger. It may be preferable to you to send an email instead of a log message (or maybe both). So, for example, you might change the logger lines above to something like this:

   echo $MSG | mail -s "`hostname | sed 's/\..*$//'` disk warning" bsdguy@fake.net

Of course, you could also do both by just adding the mail line before or after the logger line. If you use email, though, consider carefully how often you will run the job and how long it might take you to get to the machine and correct the issue. Don't spam yourself with a ton of mail! A good approach would be to break this into two scripts which are run at different frequencies and perhaps with different alerting methods (this would also make the "if" block unnecessary).

Another thing you might want to change is the level at which the script will alert. Notice that I'm just using grep to identify the high percentages. So, the part of the regular expression that says '9[5-9]' is matching any number from 95 to 99. If I wanted to change that to match anything over 80% then I'd replace it with '[89][0-9]'. If I really had to match anything at 85% and higher then I'd need an alternation and would use this: '8[5-9]\|9[0-9]'.

The regular expression '10[0-9]' is not a mistake. FreeBSD's filesystem has a reserve which means it can go over 100% utilization in df! ... Be careful cutting and pasting: the spaces inside the quotes of the regular expressions are needed. ... If you're still learning Unix, try taking apart the pipelines in the script (several commands connected by '|') and running them on the command line in parts. For example, first run "df -t ufs" then run "df -t ufs | grep ' 9[5-9]% ' ", and so on. ... This script will run fine on Linux too (either as is or with very minor modifications).



At 17/7/07 10:28, Anonymous vermaden said...

you use /usr/local/bin/bash for your scripts ( #!/usr/bin/env bash )

why you do not use pure #!/bin/sh which is optimized for script parsing?

do you use some special bash functionality or is it just a Linux habit?

At 21/7/07 00:17, Blogger kace said...

Linux habit: no!

I often do use special functionality from bash and so just use it for all of my scripts. Good point, though. This particular script looks like it could use 'sh' just fine.

At 24/7/07 14:20, Anonymous vermaden said...

"Linux habit: no!"

I used Linux (Slackware/Gentoo) before FreeBSD so I thought that you also had similar road and just got used to bash.

"I often do use special functionality from bash"

Share some example(s), I am curious about that, maybe I will find a way to use that in my scripts


At 30/7/07 00:20, Blogger kace said...

When I was new to all this a friend recommended bash (for command history, tab-expansion, etc.). That's pretty much all there was to it.

I keep looking at 'sh' to see if it has this feature or that feature and it usually does. So, I guess "often" may be an exaggeration!

But, for just a couple of nice bash-only features I've used: there's the "${parameter:offset:length}" variable expansion and (in bash 3) there's the '=~' operator to test a string against a regex.

At 31/7/07 12:05, Anonymous vermaden said...

You should try ZSH for interactive use, once you try its completion features you will never want to use antything else.

Look here for some ZSH completion examples:


Post a Comment

<< Home