3
\$\begingroup\$

I'm trying to get all files paths and names to index them in my database for faster search in future. I have used multithreading and parallel classes, but I'm couldn't get the performance I expected. There is an app named "Search Everything," which indexes all my files in less than 2 minutes, but my code takes much longer. I checked Windows Task Manager and it was my hard drive limiting my code it was getting to 100% of usage. I know that's normal, but how did "Search Everything" do that? is there another way, maybe reading data from master file table or something?

void GetFiles(string dir)
        {
            Task.Factory.StartNew(() =>
            {
                try
                {
                    files.AddRange(Directory.EnumerateFiles(dir));
                    Task.Factory.StartNew(() => {
                        Parallel.ForEach(Directory.EnumerateDirectories(dir), single => {
                            GetFiles(single);
                        });
                    });
                }
                catch { }
            });
        }
\$\endgroup\$
3
\$\begingroup\$

I am not the expert but you are creating too many threads. If possible create 5-10 threads. Whats happening here its creating too many threads and each thready try to access the resources, probably application spending too much time in sync. Try to do this. If parent directory have 5-10 sub folder(s) create one thread for each subfolder then scan subfolder using that thread only. If you have 100 subfolders under main folder, then create 10 threads and pick subfolders in 10 blocks each. I'll try to give you sample code.

| improve this answer | |
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.