# Shell script to archive files to AWS S3

This is a simple script that moves files to S3 if they are > 14 days old. It ignores files that are < 14 days old. If the file is successfully synced to AWS S3 then we can remove it from the server. If not, we leave the file where it is. The echo commands in the script are for debugging. I am aware that this could be a one liner, so interested to see what people suggest.

#!/bin/bash

fortnight=$(python -c "from datetime import datetime, timedelta; print (datetime.now()-timedelta(days=14)).strftime('%Y%m%d')") for f in /var/mail/catchall-????????.gz do if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then mv "$f" ./archive
echo "$f" echo "$(basename $f)" cd ./archive if s3cmd sync "$(basename $f)" s3://vpcgwmail ; then echo "S3 sync successful, removing file" rm "$(basename $f)" else echo "s3 sync not succesful" fi fi done  ## 1 Answer It's a fine script, but you have a bug: executing cd ./archive in a loop. Perhaps you did not notice it because you tested with only a single matching file. If there are multiple matching files, then cd ./archive will not work, unless you have multiple nested archive/archive/archive/... directories, which would be silly. One solution is to wrap the cd command and the rest of the loop body in a (...) subshell: if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then mv "$f" ./archive
echo "$f" echo "$(basename $f)" ( cd ./archive if s3cmd sync "$(basename $f)" s3://vpcgwmail ; then echo "S3 sync successful, removing file" rm "$(basename $f)" else echo "s3 sync not succesful" fi ) fi  Another solution is to not cd ./archive at all: if [[ "$f" < "/var/mail/catchall-$fortnight.gz" ]] ; then mv "$f" ./archive
echo "$f" echo "$(basename $f)" if s3cmd sync ./archive/"$(basename $f)" s3://vpcgwmail ; then echo "S3 sync successful, removing file" rm ./archive/"$(basename \$f)"
else
echo "s3 sync not succesful"
fi
fi


Whichever alternative is fine.

The indentation was also a bit off, from about the middle of the for-loop body.

• Excellent thanks for the feedback. The potential bug is a good point. Although the script will run daily, and hence will only process one file at a time (in theory!), it shouldn't encounter it. However, better safe than sorry! – kafka Aug 9 '16 at 16:20