11
\$\begingroup\$

I once asked a similar question but in C#. Now I have the same problem in powershell..

What is the fastest way, to search files newer than 15 minutes, in a file system with more than 1 million files?

Is there any faster way than using pipe?

Get-ChildItem -Path $path -Recurse | Select Name, PSIsContainer, Directory, LastWriteTime, Length| where {($_.LastWriteTime -gt (Get-Date).AddMinutes(-15))}

I already cut off some attributes to minimize the object size. It still takes ages.

\$\endgroup\$
13
\$\begingroup\$

First, you don't need to call Get-Date for every file. Just call it once at the beginning:

$t = (Get-Date).AddMinutes(-15)

Get-ChildItem -Path $path -Recurse | 
    Select Name, PSIsContainer, Directory, LastWriteTime, Length | 
    where {($_.LastWriteTime -gt $t)}

That's saves about 10% (as measured by Measure-Command).

Secondly, you don't need to call Select-Object for each file either. Just change the processing order:

$t = (Get-Date).AddMinutes(-15)

Get-ChildItem -Path $path -Recurse | 
    where {($_.LastWriteTime -gt $t)} |
    Select Name, PSIsContainer, Directory, LastWriteTime, Length

Thirdly, try increasing the buffer size using the OutBuffer parameter:

$t = (Get-Date).AddMinutes(-15)

Get-ChildItem -Path $path -Recurse -OutBuffer 1000 | 
    where {($_.LastWriteTime -gt $t)} |
    Select Name, PSIsContainer, Directory, LastWriteTime, Length

I've used 1000, but you can experiment with the value.

Those three changes reduced the running time to under one half on my system.

\$\endgroup\$
  • \$\begingroup\$ Thank you. This saves a lot of time. I know the question is about powershell. Nevertheless I found an even faster solution -> Batch. Batch does the whole thing 10 times faster. \$\endgroup\$ – greenhoorn Jan 22 '15 at 12:08
  • \$\begingroup\$ @greenhoorn, do you mean batch files (cmd.exe)? \$\endgroup\$ – Dangph Jan 22 '15 at 12:21
  • \$\begingroup\$ Yes, I do mean batch files. \$\endgroup\$ – greenhoorn Jan 22 '15 at 12:23
  • \$\begingroup\$ @greenhoorn Do you want to post your solution? Using Batch files, that is. \$\endgroup\$ – Anthony Horne Feb 7 '17 at 16:56
  • \$\begingroup\$ @AnthonyHorne This post is over 2 y old. I'm not in possession of the solution anymore. I'm sure you will find it with google like I did. \$\endgroup\$ – greenhoorn Feb 8 '17 at 7:34
1
\$\begingroup\$

https://www.avianwaves.com/Blog/entryid/177/PowerShell-Quickly-Finding-the-Oldest-and-Newest-Files-in-a-Folder.aspx

On this website, I found an example that I prefer to most of the solutions that are easily found with Google search.

The key is in using foreach-object instead of pipelining where-object filters and sort operations.

From the link above:

$newdate = [DateTime]::MinValue
$newfn = ""
$path = "."
get-childitem $path | ForEach-Object {
    if ($_.LastWriteTime -gt $newdate -and -not $_.PSIsContainer) {
        $newfn = $_.Name
        $newdate = $_.LastWriteTime
    }
}
$output = ""
if ($newfn -ne "") { $output += "`nNewest: " + $newdate + " -- " + $newfn }
if ($output -eq "") { $output += "`nFolder is empty." }
$output + "`n"

Based on everything I have read, using dir command in a batch script is still fastest option, but I dont have the motivation to learn the subtleties of batch scripting to extract the properties I want and format them as I want. PowerShell is easier to manage and GCI is great.

Using Microsoft.VisualBasic.FileIO.FileSystem may be a good alternative if you found this question. https://stackoverflow.com/questions/7196937/how-to-speed-up-powershell-get-childitem-over-unc see here for more details.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.