Check that file is fully downloaded by FTP

Question

I have a task in which I want to handle error occurred in FTP downloading, For which I used the wget command to download a file via FTP and once the download is complete, compare downloaded filesize with FTP file size.

For that, I have the code as follows. My question: is this the correct method?

# fetch file details from the server and extract only number from the return value

FileSizeOnServer=curl --max-time 10 -u $FTP_USER:$FTP_PASSWORD --head $FTP_Path 2>&1 | grep "Content-Length:"
echo "Present File Size Details:" $FileSizeOnServer

FTP_File_Size=${FileSizeOnServer#*: }
echo "FTP_File_Size:$FTP_File_Size"

# fetch only size of local file size

DownloadedFileSize=`stat -c %s /home/pi/Desktop/ABCD.txt`
echo "DownloadedFileSize:$DownloadedFileSize"

# compare local and ftp file size

if [ "$DownloadedFileSize" -eq "$FTP_File_Size" ]; then
    echo "Size matched"
else
    echo "File Size doesn't match Start Downloading again"
fi

Answer 1

You definitely need more quoting of parameter expansions. Notably, $FTP_PASSWORD may well contain characters that are meaningful to the shell:

FileSizeOnServer=$(curl --max-time 10 -u "$FTP_USER:$FTP_PASSWORD" \
                   --head "$FTP_Path" 2>&1 | grep "Content-Length:")

I'm not sure it's a good idea to redirect standard error there - you want errors to be visible, not piped into grep.

I'm not sure there's much value to the size comparison, though - curl should exit with non-zero status if it didn't transfer the whole file, so we should just use it directly:

if ! curl -u "$FTP_USER:$FTP_PASSWORD" "$FTP_Path"
then
    echo "Failed to download" >&2
    exit 1
fi

Or even simply

set -e
curl -u "$FTP_USER:$FTP_PASSWORD" "$FTP_Path"

Answer 2

I have a task in which I want to handle error occurred in FTP downloading, For which I used wget command to download a file from FTP and once the download is complete, compare downloaded filesize with FTP file size.

Your attempt is approach to solve this problem, and it's not a very good one.

The wget command exits with failure if the download isn't complete. So one better, but still naive option is to check the exit code of wget, and use the -c flag to continue downloading a partially downloaded file, for example:

while ! wget -c ...; do
     echo "Download failed, trying again..."
done

The issue you describe is quite common, and wget itself has more options to help. For example you can specify with the -t flag the number of times wget should re-try downloading a file until it succeeds.

   -t number
   --tries=number
       Set number of tries to number. Specify 0 or inf for infinite
       retrying.  The default is to retry 20 times, with the exception
       of fatal errors like "connection refused" or "not found" (404),
       which are not retried.

Other related options may also be interesting, for example --waitretry (in man wget).

---

Do not use `...`, it's obsolete. Use $(...) instead.