2
\$\begingroup\$

This is a simple script that moves files to S3 if they are > 14 days old. It ignores files that are < 14 days old. If the file is successfully synced to AWS S3 then we can remove it from the server. If not, we leave the file where it is. The echo commands in the script are for debugging. I am aware that this could be a one liner, so interested to see what people suggest.

#!/bin/bash

fortnight=$(python -c "from datetime import datetime, timedelta; print (datetime.now()-timedelta(days=14)).strftime('%Y%m%d')")

for f in /var/mail/catchall-????????.gz
do
    if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then
        mv "$f" ./archive
        echo "$f"
        echo "$(basename $f)"
        cd ./archive
            if s3cmd sync "$(basename $f)" s3://vpcgwmail ; then
                echo "S3 sync successful, removing file"
                rm "$(basename $f)"
            else
                echo "s3 sync not succesful"                
            fi
        fi
done
\$\endgroup\$
2
\$\begingroup\$

It's a fine script, but you have a bug: executing cd ./archive in a loop. Perhaps you did not notice it because you tested with only a single matching file. If there are multiple matching files, then cd ./archive will not work, unless you have multiple nested archive/archive/archive/... directories, which would be silly.

One solution is to wrap the cd command and the rest of the loop body in a (...) subshell:

if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then
    mv "$f" ./archive
    echo "$f"
    echo "$(basename $f)"
    (
        cd ./archive
        if s3cmd sync "$(basename $f)" s3://vpcgwmail ; then
            echo "S3 sync successful, removing file"
            rm "$(basename $f)"
        else
            echo "s3 sync not succesful"                
        fi
    )
fi

Another solution is to not cd ./archive at all:

if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then
    mv "$f" ./archive
    echo "$f"
    echo "$(basename $f)"
    if s3cmd sync ./archive/"$(basename $f)" s3://vpcgwmail ; then
        echo "S3 sync successful, removing file"
        rm ./archive/"$(basename $f)"
    else
        echo "s3 sync not succesful"                
    fi
fi

Whichever alternative is fine.

The indentation was also a bit off, from about the middle of the for-loop body.

\$\endgroup\$
  • \$\begingroup\$ Excellent thanks for the feedback. The potential bug is a good point. Although the script will run daily, and hence will only process one file at a time (in theory!), it shouldn't encounter it. However, better safe than sorry! \$\endgroup\$ – kafka Aug 9 '16 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.