This code is to gather a sorted list of IDs (first by ID, then by date) recursively from a directory that submitted a particular type of file (that, as you can tell, includes the word "Magic") and then outputs the IDs paired with the modified date (the closest proxy I have for the submission date) to a CSV. (Where I then dedupe by ID to get the earliest submission for each ID/find average number of unique IDs submitted a month over the year/ the total submitted for the year):
Get-ChildItem -Recurse -Include *Magic* |
Where-Object {$_.lastWriteTime -gt '01/01/2018' -AND $_.lastWriteTime -lt '12/31/2018'} |
Select-Object -Property Directory, lastWriteTime |
ForEach-Object -Begin {
$holder=@{}
$results=@()
} -Process {
$holder.id= [regex]::match($_.Directory,'(\d{8})')
$holder.modifiedDate = Get-Date $_.lastWriteTime -Format 'yyyy/MM/dd'
$results+=[pscustomobject]$holder
} -End {
$results
} |
Sort-Object -Property @{Expression = "id"; Descending = $True;}, @{Expression = "modifiedDate"; Descending = $False} |
Export-csv C:\Users\REDACTED\Desktop\REDACTED.csv
This is my first attempt at writing something in powershell purely using the command prompt (which is to say, not scripting in the ISE as if it were Ruby using ifs and so forth) How could I have done this better and should I have organized the pipeline differently? I tried to avoid using variables like would have in the past and focused purely on piping the data I was interested in from beginning to end.
EDIT
Based on the comment I got I made these changes. I wasn't able to get rid of all the pipelines and I haven't been able to test if it's faster or not, but I have a feeling even this version has some garbage that could go.
$sourceFiles = Get-ChildItem -Recurse -Include *Magic*
$filteredFiles = @()
ForEach($file in $sourceFiles){
If($file.LastWriteTime.Year -eq 2018){
$filteredFiles+=[PSCustomObject]@{
Agency = [regex]::match($file.Directory,'(\d{8})')
ModifiedDate = Get-Date $file.lastWriteTime -Format 'yyyy/MM/dd'
}
}
}
$filteredfiles | Sort-Object -Property @{Expression = "Agency"; Descending =
$True;}, @{Expression = "ModifiedDate"; Descending = $False} |
Export-csv C:\Users\REDACTED\Desktop\REDACTED.csv -NoTypeInformation
$_.LastWriteTime.Year -eq 2018. otherwise, things seem OK. \$\endgroup\$ – Lee_Dailey Jan 26 '19 at 2:36$Holderhashtable seems unneeded. plus, it seems likely to give you nearly random property order. just create the items in the[PSCustomObject]call. ///// yet another - your$resutlscollection seems unneeded. simply put the[PSCustomObject]call on it's own line to send that to the pipeline. \$\endgroup\$ – Lee_Dailey Jan 26 '19 at 2:40$sourceFiles = Get-ChildItem -Recurse -Include *Magic* $filteredFiles = Where-Object {$sourceFiles.lastWriteTime.Year -eq 2018}and then found a way to get the ordered pairs of directories and dates from the files? I was using the pipeline because I was patterning things after a tutorial I watched, but didn't realize it was the reason everything was so incredibly slow \$\endgroup\$ – jasonmadesomething Jan 28 '19 at 16:46-Includewith-Filter, then add-Fileto ensure i only got back files, and last would explicitly indicate the path with either-LiteralPathor-Path. the 1st will handle embedded[]chars, the 2nd allows wildcards. ///// next i would remove the+=since adding to arrays is SLOW. simply use$Result = foreach ($Thing in $Collection)to assign the whole set of objects to the collection all at once. ///// last i would use a[PSCustomObject]for object-building instead of building a new object withSelect-Object. a PSCO structure is usually easier to read. \$\endgroup\$ – Lee_Dailey Jan 28 '19 at 18:25