I created a small console application to recursively search a folder and all subfolders for files that contain a match for a regex.
The command line arguments are the path you're looking to search followed by the regex (in .NET regex format).
I break this up into two threads: the main thread loops through the files and puts file data (their content, path, and name) on a concurrent queue. The second thread processes the queue, looking for regex matches.
Would using a Parallel.ForEach
loop be better here? The main issue I had with this is that the queue is being modified while we're looping over it, but a Parallel.ForEach
loop assumes that you already have a complete collection. The two possibilities I came up with this:
- Just read the content of the files, and then do a
Parallel.ForEach
loop to do the regex. - Process it on the background thread like I'm doing until we're done reading the file contents, then do a
Parallel.ForEach
loop over any remaining items once we're done.
What I have here works but is slower than I'd like. I'd definitely like feedback on the multithreading I'm doing here, especially if one of the possibilities I mentioned above would be better than what I'm doing (or if there's another possibility that I don't mention above).
namespace SearchByRegex
{
class Program
{
// I should probably just move this out to its own file
private class FilePair
{
public string Content { get; set; }
public string FilePath { get; set; }
public string FileName { get; set; }
}
private static ConcurrentQueue<FilePair> fileInformation = new ConcurrentQueue<FilePair>();
static void Main(string[] args)
{
Console.WriteLine("Working...");
// Command lines are a path followed by the regular expression
var regex = new Regex(args[1], RegexOptions.Multiline);
// Process
Thread thread = new Thread(PrintInformationOnMatches);
thread.Start(regex);
SearchDirectory(args[0]);
while (fileInformation.Any())
{
Thread.Sleep(1000);
}
thread.Abort();
Console.WriteLine("Press any key to exit");
Console.ReadLine();
}
/// <summary>
/// Print information on all files whose content matches the <paramref name="regex"/>
/// </summary>
/// <param name="regex">Regex we're looking for</param>
private static void PrintInformationOnMatches(object regex)
{
var castRegex = (Regex)regex;
while (true)
{
// Process the entire queue
while (fileInformation.Any())
{
FilePair pair;
fileInformation.TryDequeue(out pair);
if (castRegex.IsMatch(pair.Content))
{
Console.WriteLine($"File name: {pair.FileName}. Path: {pair.FilePath}.");
}
}
// Wait for more stuff on the queue
Thread.Sleep(1000);
}
}
/// <summary>
/// Get the contents of the file and put them on the queue
/// </summary>
/// <param name="path">Path we are recursively searching</param>
private static void SearchDirectory(string path)
{
string[] files = Directory.GetFiles(path);
//Console.WriteLine("Path: " + path);
foreach (string file in files)
{
fileInformation.Enqueue(new FilePair { Content = File.ReadAllText(file), FilePath = path, FileName = file });
}
foreach (string directory in Directory.GetDirectories(path))
{
SearchDirectory(directory);
}
}
}
}