I created this function to recursively copy an entire directory from an FTP server. It works just fine except that it is about 4 times slower than using FileZilla to do the same operation. It takes approximately 55 seconds to download the directory in FileZilla but it takes 229 seconds with this function. What can I do to make it download/run faster?

Private Sub CopyEntireDirectory(ByVal directory As String)
Dim localPath = localDirectory & formatPath(directory)
'creates directory in destination path
IO.Directory.CreateDirectory(localPath)

'Gets the directory details so I can separate folders from files
Dim fileList As ArrayList = Ftp.ListDirectoryDetails(directory, "")

For Each item In fileList
'checks if it's a folder or file: d=folder
If (item.ToString().StartsWith("d")) Then
'gets the directory from the details
Dim subDirectory As String = item.ToString().Substring(item.ToString().LastIndexOf(" ") + 1)
CopyEntireDirectory(directory & "/" & subDirectory)
Else
Dim remoteFilePath As String = directory & "/" & item.ToString().Substring(item.ToString().LastIndexOf(" ") + 1)
Dim destinationPath = localPath & "\" & item.ToString().Substring(item.ToString().LastIndexOf(" ") + 1)
End If
Next
End Sub


This function below is what is actually taking up 97% of the time.

Public Sub DownLoadFile(ByVal fromFilename As String, ByVal toFilename As String)
Dim files As ArrayList = Me.ListDirectory(fromFilename, "")
Dim request As FtpWebRequest = Me.CreateRequestObject(fromFilename)

Dim response As FtpWebResponse = CType(request.GetResponse(), FtpWebResponse)
If response.StatusCode <> FtpStatusCode.OpeningData AndAlso response.StatusCode <> FtpStatusCode.DataAlreadyOpen Then
Throw New ApplicationException(Me.BuildCustomFtpErrorMessage(request, response))
End If

Dim fromFilenameStream As Stream = response.GetResponseStream()
Dim toFilenameStream As FileStream = File.Create(toFilename)

Dim buffer(BLOCK_SIZE) As Byte
Array.Clear(buffer, 0, buffer.Length)
Loop

response.Close()
fromFilenameStream.Close()
toFilenameStream.Close()
End Sub

• FileZilla is probably using threads to download multiple files at the same time. You may want to consider first creating the directory structure and a list of files with full paths to download, then using the Task Parallel Library to split that list in to multiple download jobs. Specifically, Parallel.ForEach msdn.microsoft.com/en-us/library/dd460720(v=vs.110).aspx – Bradley Uffner Dec 2 '15 at 20:28
• It appears that FileZilla sets up a queue and downloads the files one after another. – Programmer Dec 2 '15 at 20:30
• I wouldn't know how this compares speedwise, but I've previously found this useful: stackoverflow.com/a/7535879/1316573 – Daniel Dec 4 '15 at 14:59
• @BradleyUffner I am using Parallel.ForEach but there is a hangup when it uses Me.ListDirectory(fromFilename, ""). Somewhere it doesn't allow Parallel Tasks and is blocking the operation. – Programmer Dec 14 '15 at 17:14