7
\$\begingroup\$

I am sick of File Explorer's super slow deletion speed, so I tried to write a PowerShell script to make deletion faster, and while it does its job, its speed isn't as high as what I intended it to be.

I have written this:

Function Fast-Delete {
    Param(
    [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory0
    )
    Write-Warning "This Process Will Delete Everything In The Target Directory: $Directory0, Do You Want To Confirm Deletion?" -Warningaction Inquire
    $Directory=$Directory0
    While ((Get-Childitem -Path $Directory0 -Depth 0 -Force).Count -Ne 0) {
    If ((Get-Childitem -Path $Directory -Directory -Force -Depth 0).Count -Ne 0) {
    $Directory=(Get-Childitem -Path $Directory -Directory -Force -Depth 0).Fullname | Select-Object -Index 0
    }
    If ((Get-Childitem -Path $Directory -File -Force).Count -Ne 0) {
    (Get-Childitem -Path $Directory -File -Recurse -Force).Fullname | Foreach {Remove-Item -Path $_}
    }
    $Directory1=$Directory
    $Directory=$Directory | Split-Path -Parent
    Remove-Item -Path $Directory1
    }
    Remove-Item -Path $Directory0
}

It is significantly faster than explorer, but still isn't ideal, I have tested it, I used it to delete 208,000 files in 1,000 folders , and the folders disappear at speed of 1 per second, so it's about 208 files/s, now the next challenge should be parallelization, But this is currently really way above me, but it shouldn't be too hard, I am just not experienced enough.

Update2

I have managed to make it run in parallel with this script:

function Parallel-Delete {
    param(
    [Parameter(Valuefrompipeline=$true, Mandatory=$true, Position=0)] [array]$filelist,
    [Parameter(Valuefrompipeline=$true, Mandatory=$true, Position=1)] [int]$number
    )
    0..($filelist.count-1) | Where-Object {$_ % 16 -eq $number} | foreach {Remove-Item -Path $filelist[$_]}
}

Function Fast-Delete {
    Param(
    [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory0
    )
    Write-Warning "This Process Will Delete Everything In The Target Directory:  $Directory0, Do You Want To Confirm Deletion?" -Warningaction Inquire
    $Directory=$Directory0
    While ((Get-Childitem -Path $Directory0 -Depth 0 -Force).Count -Ne 0) {
        If ((Get-Childitem -Path $Directory -Directory -Force -Depth 0).Count -Ne 0) {
            $Directory=(Get-Childitem -Path $Directory -Directory -Force -Depth 0).Fullname | Select-Object -Index 0
        }
        If ((Get-Childitem -Path $Directory -File -Force).Count -Ne 0) {
            If ((Get-Childitem -Path $Directory -File -Force).Count -Ge 128) {
                [array]$filelist=(Get-Childitem -Path $Directory -File -Force).Fullname
                0..15 | foreach-object {Invoke-Command -ScriptBlock { Parallel-Delete $filelist $_}}
                } else {
                (Get-Childitem -Path $Directory -File -Force).Fullname | Foreach {Remove-Item -Path $_}
            }
        }
        $Directory1=$Directory
        $Directory=$Directory | Split-Path -Parent
        Remove-Item -Path $Directory1
        }
    Remove-Item -Path $Directory0 -erroraction silentlycontinue
}
$Directory0=Read-Host "Please input target directory to be deleted"
Fast-Delete $Directory0

But it still isn't ideal, it isn't 15 times faster as expected, what did I miss?

Edit: simplified the creation of parallel processes.

\$\endgroup\$
11
  • 2
    \$\begingroup\$ the fastest way to delete an entire dir tree is to use robocopy. you mirror a blank "source" dir to the one you want removed ... and the "destination" gets emptied quickly. [grin] \$\endgroup\$
    – Lee_Dailey
    Commented Dec 9, 2020 at 7:33
  • \$\begingroup\$ @Lee_Dailey, in fact I wanted to write a new script, not to use existing things, and the method you mentioned sounds like a bug... \$\endgroup\$ Commented Dec 9, 2020 at 7:51
  • 3
    \$\begingroup\$ i understand your desire. [grin] as for a bug ... it is exactly what the options mean - mirror empty source to full destination gives empty source and empty destination. so ... have fun playing with a script ...but PoSh will always be slower than any well written utility. \$\endgroup\$
    – Lee_Dailey
    Commented Dec 9, 2020 at 8:08
  • 1
    \$\begingroup\$ why you dont remove the dire complete and recreate him after ? \$\endgroup\$ Commented Apr 25, 2021 at 5:48
  • 1
    \$\begingroup\$ So... I've been doing some testing, as I'm in a similar boat. A 2 GB folder, containing 10,000 items, is stored on a network drive (I had ~200 when I started removing directories). An individual directory click-delete via explorer has been deleting 3.1 files/sec (includes analyze and deletiion time); a single line remove-item I average 2.4 files/sec; a power shell script for robocopy /purge with a blank directory averages 2.4 files/sec (this still analyzes each individual file in the parent directory)... i am about to try robocopy /mir to see if it's any faster. \$\endgroup\$
    – Cyril
    Commented Aug 19, 2022 at 17:14

2 Answers 2

5
\$\begingroup\$

As pointed out in a comment from @Lee_Daily, reinventing the wheel from an interpreted language like PowerShell will be slower than a native implementation. If your primary concern is speed, use the correct tool for the job. We have a native implementation in robocopy.

It has an option /mir

Mirrors a directory tree (equivalent to /e plus /purge). Using this option with the /e option and a destination directory, overwrites the destination directory security settings.

Below is the robocopy purge method using your framework:

Param(
    [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory
)

Function Fast-Delete {
    Param(
        [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory
    )

    Write-Warning "This Process Will Delete Everything In The Target Directory:  $Directory, Do You Want To Confirm Deletion?" -Warningaction Inquire

    $emptyDir = GetRandomTempPath
    mkdir $emptyDir

    robocopy $emptyDir $Directory /mir

    Remove-Item -Path $Directory -erroraction silentlycontinue
    Remove-Item -Path $emptyDir -erroraction silentlycontinue
}

Function GetRandomTempPath {
    Do {
        $fullPath = Join-Path -Path $env:TEMP -ChildPath [System.IO.Path]::GetRandomFileName()
    } While (Test-Path $fullPath);

    Return $fullPath;
} 

Fast-Delete $Directory

Save as: Fast-Delete.ps1

Comparison of delete methods (2 GB, 10,000 files in directory):

  • remove-item: 2.4 files/sec average (n=10)
  • /purge : 2.4 files/sec average (n=3)
  • /purge /mt: 2.5 files/sec average (n=3)
  • explorer-based click-delete: 3.1 files/sec (n=30)
  • /mir: 5.5 files/sec (n = 100)
\$\endgroup\$
2
  • \$\begingroup\$ I don't use PowerShell myself (I'm more of a POSIX shell and Bash user), but it looks to my untrained eye that your suggested code assumes that there's no existing non-empty $TEMP/uniqueemptydirname, without checking. Is there no mktemp available from this shell? Also, I'd guess that the most useful amount of parallelism would depend more on the storage throughput than on the number of CPU threads here. \$\endgroup\$ Commented Aug 20, 2022 at 8:47
  • \$\begingroup\$ You are 100% correct in that assumption. I added a function for GetRandomTempPath in PowerShell. It doesn't come with a method like this that I could find. \$\endgroup\$
    – HackSlash
    Commented Aug 22, 2022 at 15:45
3
\$\begingroup\$

The foreach-object is not run in parallel by default. You need specify the -Parallel parameter. And the version of Powershell need be greater than 7.0

See Official Blob and Official document

\$\endgroup\$

Not the answer you're looking for? Browse other questions tagged or ask your own question.