Timeline for PowerShell - fast remove a directory with 10,000+ files
Current License: CC BY-SA 4.0
34 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 22, 2023 at 14:11 | history | protected | Mast♦ | ||
| Aug 22, 2022 at 15:19 | comment | added | Cyril |
I finished removing all of my back-up directories using HacknSlash's code, changing from /purge to /mir. That change went from ~2.4 files/sec to ~5.5 files/sec (n=100). It was the fastest method of those I tried. Your speed may be drastically different as the only way to access this network drive is via VPN, so that connection was a limiting factor, which I feel allowed a more full picture of the differences in speed to execute each method.
|
|
| Aug 19, 2022 at 17:15 | comment | added | Cyril | I have found the best success thusfar individually deleting files via explorer (typically 10 at a time), where I drop to roughly the same delete speed as individual directory removal via power shell. The in-parallel deletion has given me a small/minimal drop at moving from 1 directories to 5 directories, and an almost 1 file/sec drop at 10, where each deleted file is ~10k files inside of a 2 GB directory. | |
| Aug 19, 2022 at 17:14 | comment | added | Cyril |
So... I've been doing some testing, as I'm in a similar boat. A 2 GB folder, containing 10,000 items, is stored on a network drive (I had ~200 when I started removing directories). An individual directory click-delete via explorer has been deleting 3.1 files/sec (includes analyze and deletiion time); a single line remove-item I average 2.4 files/sec; a power shell script for robocopy /purge with a blank directory averages 2.4 files/sec (this still analyzes each individual file in the parent directory)... i am about to try robocopy /mir to see if it's any faster.
|
|
| Aug 18, 2022 at 17:02 | answer | added | HackSlash | timeline score: 5 | |
| Jun 24, 2022 at 9:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Feb 24, 2022 at 8:08 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jan 25, 2022 at 7:12 | answer | added | Enpeng | timeline score: 3 | |
| Jan 20, 2022 at 8:05 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Sep 22, 2021 at 8:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| May 25, 2021 at 7:08 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Apr 25, 2021 at 5:48 | comment | added | Esperento57 | why you dont remove the dire complete and recreate him after ? | |
| Feb 4, 2021 at 6:00 | history | tweeted | twitter.com/StackCodeReview/status/1357207273542672386 | ||
| Feb 4, 2021 at 0:17 | comment | added | @XeнεiΞэnвϵς Yes, it's me from the other sister site with an old handle name or whatever it is called. Look at my profile and click on the SU community from there and you'll see. I don't use the same avatar or moniker on each of the communities always so I mix and match like a bucket of chicken from KFC. | ||
| Feb 4, 2021 at 0:14 | comment | added |
@XeнεiΞэnвϵς Replace all where-object with foreach-object and use If() conditions instead. Put process { } wrap in all your foreach-object and also in all invoke-command within your script. E.g. 0..($filelist.count-1) | % { Process { If ($_ % 16 -eq $number) { Remove-Item -Path $filelist[$_] } } } and 0..15 | % { Process { Invoke-Command -ScriptBlock { Process { Parallel-Delete $filelist $_ }}}} for starters... Just start incorporating that way and then measure to test the speed differences. I always start with those simple techniques for PS processing speed 101.
|
||
| Dec 16, 2020 at 7:12 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
deleted 899 characters in body
|
| Dec 15, 2020 at 23:37 | comment | added | Lee_Dailey |
i would narrow your problems to specifics ... and open a Question for each of them. the pass variable to start-process question has been answered here OR in stackoverflow many times.
|
|
| Dec 15, 2020 at 10:37 | comment | added | Ξένη Γήινος | And I have tried Start-Job, but they just hang in place, they are not working so I used stop-job to terminate them... And still I can't pass to variables to new powershell process started by start-process... | |
| Dec 15, 2020 at 10:31 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
deleted 1263 characters in body
|
| Dec 10, 2020 at 6:03 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
deleted 1796 characters in body
|
| Dec 9, 2020 at 12:48 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 1 character in body
|
| Dec 9, 2020 at 12:39 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 8 characters in body
|
| Dec 9, 2020 at 12:17 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 2533 characters in body
|
| Dec 9, 2020 at 9:49 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 2 characters in body
|
| Dec 9, 2020 at 9:20 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 20 characters in body
|
| Dec 9, 2020 at 9:10 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 2 characters in body
|
| Dec 9, 2020 at 9:03 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 2 characters in body
|
| Dec 9, 2020 at 8:56 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 2201 characters in body
|
| Dec 9, 2020 at 8:08 | comment | added | Lee_Dailey | i understand your desire. [grin] as for a bug ... it is exactly what the options mean - mirror empty source to full destination gives empty source and empty destination. so ... have fun playing with a script ...but PoSh will always be slower than any well written utility. | |
| Dec 9, 2020 at 8:08 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
added 11 characters in body
|
| Dec 9, 2020 at 7:51 | comment | added | Ξένη Γήινος | @Lee_Dailey, in fact I wanted to write a new script, not to use existing things, and the method you mentioned sounds like a bug... | |
| Dec 9, 2020 at 7:33 | comment | added | Lee_Dailey | the fastest way to delete an entire dir tree is to use robocopy. you mirror a blank "source" dir to the one you want removed ... and the "destination" gets emptied quickly. [grin] | |
| Dec 9, 2020 at 7:15 | history | edited | Ξένη Γήινος | CC BY-SA 4.0 |
edited body
|
| Dec 9, 2020 at 7:06 | history | asked | Ξένη Γήινος | CC BY-SA 4.0 |