11
\$\begingroup\$

I once asked a similar question but in C#. Now I have the same problem in powershell..

What is the fastest way, to search files newer than 15 minutes, in a file system with more than 1 million files?

Is there any faster way than using pipe?

Get-ChildItem -Path $path -Recurse | Select Name, PSIsContainer, Directory, LastWriteTime, Length| where {($_.LastWriteTime -gt (Get-Date).AddMinutes(-15))}

I already cut off some attributes to minimize the object size. It still takes ages.

\$\endgroup\$

2 Answers 2

16
\$\begingroup\$

First, you don't need to call Get-Date for every file. Just call it once at the beginning:

$t = (Get-Date).AddMinutes(-15)

Get-ChildItem -Path $path -Recurse | 
    Select Name, PSIsContainer, Directory, LastWriteTime, Length | 
    where {($_.LastWriteTime -gt $t)}

That's saves about 10% (as measured by Measure-Command).

Secondly, you don't need to call Select-Object for each file either. Just change the processing order:

$t = (Get-Date).AddMinutes(-15)

Get-ChildItem -Path $path -Recurse | 
    where {($_.LastWriteTime -gt $t)} |
    Select Name, PSIsContainer, Directory, LastWriteTime, Length

Thirdly, try increasing the buffer size using the OutBuffer parameter:

$t = (Get-Date).AddMinutes(-15)

Get-ChildItem -Path $path -Recurse -OutBuffer 1000 | 
    where {($_.LastWriteTime -gt $t)} |
    Select Name, PSIsContainer, Directory, LastWriteTime, Length

I've used 1000, but you can experiment with the value.

Those three changes reduced the running time to under one half on my system.

\$\endgroup\$
6
  • \$\begingroup\$ Thank you. This saves a lot of time. I know the question is about powershell. Nevertheless I found an even faster solution -> Batch. Batch does the whole thing 10 times faster. \$\endgroup\$
    – greenhoorn
    Jan 22, 2015 at 12:08
  • \$\begingroup\$ @greenhoorn, do you mean batch files (cmd.exe)? \$\endgroup\$
    – Dangph
    Jan 22, 2015 at 12:21
  • \$\begingroup\$ Yes, I do mean batch files. \$\endgroup\$
    – greenhoorn
    Jan 22, 2015 at 12:23
  • \$\begingroup\$ @greenhoorn Do you want to post your solution? Using Batch files, that is. \$\endgroup\$ Feb 7, 2017 at 16:56
  • \$\begingroup\$ @AnthonyHorne This post is over 2 y old. I'm not in possession of the solution anymore. I'm sure you will find it with google like I did. \$\endgroup\$
    – greenhoorn
    Feb 8, 2017 at 7:34
2
\$\begingroup\$

https://www.avianwaves.com/Blog/entryid/177/PowerShell-Quickly-Finding-the-Oldest-and-Newest-Files-in-a-Folder.aspx

On this website, I found an example that I prefer to most of the solutions that are easily found with Google search.

The key is in using foreach-object instead of pipelining where-object filters and sort operations.

From the link above:

$newdate = [DateTime]::MinValue
$newfn = ""
$path = "."
get-childitem $path | ForEach-Object {
    if ($_.LastWriteTime -gt $newdate -and -not $_.PSIsContainer) {
        $newfn = $_.Name
        $newdate = $_.LastWriteTime
    }
}
$output = ""
if ($newfn -ne "") { $output += "`nNewest: " + $newdate + " -- " + $newfn }
if ($output -eq "") { $output += "`nFolder is empty." }
$output + "`n"

Based on everything I have read, using dir command in a batch script is still fastest option, but I dont have the motivation to learn the subtleties of batch scripting to extract the properties I want and format them as I want. PowerShell is easier to manage and GCI is great.

Using Microsoft.VisualBasic.FileIO.FileSystem may be a good alternative if you found this question. https://stackoverflow.com/questions/7196937/how-to-speed-up-powershell-get-childitem-over-unc see here for more details.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.