Alerting for rapid disk growth

Question

I have a server that is pretty stable (disk capacity) until a certain event occurs and then the drive grows over the course of two days until it is full. I wrote a very simple bash script but I'm not too happy with parts of it. First, I want to be able to see the capacity as a percent down to the tenths place. Also, doing an evaluation I had to strip my value of any decimal places to compare it against my defined threshold. Lastly I feel that I made too much use of cut where I could've been more efficient. I'd love to get some tips for improvement.

#! /bin/bash

monDrive=/opt
prevReading=/root/diskcheckreading.txt
percentThreshold=1
[email protected]

#Check if log file exists

if [ ! -f $prevReading ]
    then
        touch $prevReading
        prev=$(printf "%0.2f\n" $(bc -l <<< $(df /opt | grep opt | cut -d' ' -f24)/$(df /opt | grep opt | cut -d' ' -f23)*100))
        echo $prev > $prevReading

fi


#Get current percentage used of monitored drive
current=$(printf "%0.2f\n" $(bc -l <<< $(df /opt | grep opt | cut -d' ' -f24)/$(df /opt | grep opt | cut -d' ' -f23)*100))

#import previous reading
prev=$(cat $prevReading)

#Calculate the difference
results=$(bc -l <<< $current-$prev )

#Convert to percentage
results100=$(bc <<< $results*100)

#Remove any number after decimal place so that bash can evaluate
rescleanup=$(echo $results100 | cut -d'.' -f1)

#Convert to percentage
percentThreshold100=$(bc -l <<< $percentThreshold*100)

#Check if the difference exceeds the threshold
if  [ $rescleanup -gt $percentThreshold100 ]
     then printf "\"$monDrive\" has grown $results%% in the past hour.\n\n$(hostname):$(pwd)/$(basename $0)" | mail -s "Disk Alert - $(hostname):$monDrive" $emailAlert

fi

#log current reading for next check
emailecho $current > $prevReading

Answer 1

1. Proceed when prevReading file does not exist?

   Your script proceeds to create the file first, and then almost immediately performs the same calculation. In this case, it might be better to just create the file first, exit from the script and then wait for its next invocation to do the comparison.

2. Performing the same calculation twice

   You are repeating the same commands for setting prev and current, so you should put it in a function:

   checkDiskUse() {
       df "$monDrive" | awk '/\'"$monDrive"'/{printf "%0.2f",$3/$2*100; exit}'
   }
   

3. cut -d' ' -f23/24 can be substituted with awk

   As illustrated above, awk can 'solve' your weird usage of cut-ting on whitespaces, and then calling bc on them. Your use of cut assumes that the relevant line has your desired values (used / total) exactly 23 whitespaces before them, which may be wrong in the future. Let awk do the whitespace parsing for you instead, and simply extract the third and second values respectively ($3, $2).

4. Validate file contents

   It's good practice to sanity check that the file you are reading from has valid data. A simple way to do so is to check if printf is able to parse it correctly or not:

   printf "%0.2f" "$prev" > /dev/null || exit
   

   If the input value $prev is valid, printf returns with a successful exit code. Otherwise, the statement proceeds to the 'or' || part and exit the script as a fail-safe measure.

5. [ vs [[

   As you have discovered, the basic test command [ does not deal with decimal values, but bash's built-in [[ conditional expression happens to handle that well. Compare:

   $ [ 0.601 -gt 0.6 ] && echo Y
   bash: [: 0.601: integer expression expected
   $ [[ 0.601 > 0.6 ]] && echo Y
   Y
   

---

To sum it up

monDrive=/opt
prevReading=/root/diskcheckreading.txt
percentThreshold=1
[email protected]
checkDiskUse() {
    df "$monDrive" | awk '/\'"$monDrive"'/{printf "%0.2f",$3/$2*100; exit}'
}
[ -f $prevReading ] && prev=$(cat $prevReading) || { checkDiskUse > $prevReading; exit; }
printf "%0.2f" "$prev" > /dev/null || exit
current=$(checkDiskUse)
results=$(bc <<< "$current - $prev")
if [[ $results > $percentThreshold ]]
    then printf "\"$monDrive\" has grown $results%% in the past hour.\n\n$(hostname):$(pwd)/$(basename $0)" | \
        mail -s "Disk Alert - $(hostname):$monDrive" $emailAlert
fi
emailecho $current > $prevReading

Note: $monDrive is assumed to only contain / at the start, since only the first is escaped in the awk statement. Refer to comments for a somewhat hack-y workaround in vim's syntax highlighting (YMMV). Alternatively, I think using END instead of the regex might work too:

df "$monDrive" | awk 'END{printf "%0.2f",$3/$2*100; exit}'

Answer 2

• bc is an overkill. For simple calculations use arithmetic expansion, e.g.

  results = $(($current - $prev))
  

• read built-in is way more friendly than grep/cut combo:

  df /opt | (read; read fs blocks used available rest; compute_percentage $used $available)
  

Answer 3

Based on the help received here, this is the updated version that is tested and working. Thanks for the help!

#!/bin/bash 

monDrive=/opt
prevReading=/root/diskcheckreading.txt
percentThreshold=1
[email protected]

checkDiskUse() {
        df "$monDrive" | awk 'END{printf "%0.2f",$2/$1*100; exit}'
}

#Check if file exists with previous reading
[ -f $prevReading ] && prev=$(cat $prevReading) || { checkDiskUse > $prevReading; exit; }

#Validate file content
printf "%0.2f" "$prev" > /dev/null || exit

#Get current percentage used of monitored drive
current=$(checkDiskUse)

#Calculate the difference
results=$(bc -l <<< $current-$prev )

#Check if the difference exceeds the threshold
if  [[ $results > $percentThreshold ]]
     then printf "\"$monDrive\" has grown $results%% in the past hour.\n\n$(hostname):$(pwd)/$(basename $0)" | mail -s "Disk Alert - $(hostname):$monDrive" $emailAlert

fi

#log current reading for next check
echo $current > $prevReading