Program that deletes near duplicate directories and directories that have .ini like files

Question

The program needs to only keep directories that have at least 1 .osu (.ini like) file with certain restrictions in them. It also needs to delete duplicate .osu files. But since duplicate files are only in duplicate directories (never in the same directory), we can also just delete all but one directories with a similar name. If you know a better way of achieving this, I'd love to hear it though

I currently achieve it by:

• look at first few indexes of directory name
  • if they match any of the first few indexes of the other directory names, delete them
• read every .osu file line by line
  • if they contain x keyword and x value, set correctFile boolean true
  • delete every file where the boolean is false
• delete every directory that doesn't contain an .osu file

My code:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace DeleteAllUnnecesaryOsuBeatmaps
{
    class Program
    {
        static void Main(string[] args)
        {
            // variables //
            string path = @"D:\Games\New folder\Songs";  // change to your songs folder
            List<string> thingsToDel = new List<string>();



            // delete duplicate directories //
            string[] allDirs = Directory.GetDirectories(path);
            // the amount of characters (counting from the beginning of the dir name) that have to match before it's counted as a duplicate
            int indexesToCheck = 8;

            for (int i = 0; i < allDirs.Length; i++)
            {
                // makes a substring with 'indexesToCheck' amount of characters from the directory name
                string dirStart = allDirs[i].Substring(path.Length + 2, indexesToCheck); 
                // the +2 is because the path allDirs[i] has an additional '//'

                // goes trough every dir above i and compares it to 'dirStart'
                for (int j = i + 1; j < allDirs.Length; j++)
                {
                    // makes another substring
                    string otherDirStart = allDirs[j].Substring(path.Length + 2, indexesToCheck);
                    // if they match, the dir of index i will be deleted later on
                    if (dirStart == otherDirStart)
                    {
                        thingsToDel.Add(allDirs[i]);
                        break;
                    }
                }
            }

            // deletes everything in 'thingsToDel'
            foreach (string thing in thingsToDel)
            {
                Directory.Delete(thing, true);
                // the true stands for deleting directories, even if they have subdirectories/files
            }
            thingsToDel.Clear();



            // delete files that do not match my needs //
            string[] allFiles = Directory.GetFiles(path, "*.osu", SearchOption.AllDirectories);
            // .osu files are comparable to .ini files

            // if true, the file will not be deleted
            bool fileIsCorrect = false;
            // if true, the file has mania as mode and needs an aditional check
            bool modeIsMania = false;

            // checks for each file, if it should be deleted
            foreach (string file in allFiles)
            {
                fileIsCorrect = false;
                modeIsMania = false;
                string[] lines = File.ReadAllLines(file);

                foreach (string line in lines)
                {
                    // format is: Mode = (number between 0 and 3)
                    if (line.Contains("Mode"))
                    {
                        if (line.Contains("0"))  // change to your gamemode/s (add || line.Contains("x") to add one)
                        {
                            fileIsCorrect = true;
                        }
                        else if (line.Contains("3"))  // delete these 4 rows if you don't want mania maps
                        {
                            modeIsMania = true;
                        }
                    }
                    else if (modeIsMania && (line.Contains("CircleSize") && (line.Contains("4") || line.Contains("7") || line.Contains("10"))))
                    { // circlesize is an additional check that only mode mania/3 needs
                        fileIsCorrect = true;
                    }
                }

                if (!fileIsCorrect)
                {
                    thingsToDel.Add(file);
                }
            }

            foreach (string thing in thingsToDel)
            {
                File.Delete(thing);
            }
            thingsToDel.Clear();



            // delete directories without .osu files //
            allDirs = Directory.GetDirectories(path);

            foreach (string dir in allDirs)
            {
                // if a dir does not have at least 1 .osu file in it, it will be delete later
                if (Directory.GetFiles(dir, "*.osu").Length == 0)
                {
                    thingsToDel.Add(dir);
                }
            }

            foreach (string thing in thingsToDel)
            {
                Directory.Delete(thing, true);
            }
            thingsToDel.Clear();



            // end of program //
            Console.WriteLine(
                "The Program is finished!\n" +
                "Press any key to exit...");
            Console.ReadKey();
        }
    }
}

test environment: https://drive.google.com/file/d/15ipqiXoUTeC5s7KLgzPi1Pmht-fbfLcz/view?usp=sharing .osu file example: https://drive.google.com/file/d/174SCGPA3fM47SxoNs7dnbQZRQp51mmJs/view?usp=sharing

the problem is that most users will delete up to thousands of directories with each an average of 3 .osu files in them so it NEEDS to be efficient. If you know of any way to optimize my methode or the functions I use/my code, please let me know!

Edit: I'm still a beginner so any tips about structure, comments, naming, expandability etc. would be much appreciated as well!

Edit 2: Added a test environment and .osu file example

Answer 1

Look into using HashSets. If you want to see if an item is already in a list, you have to search through thousands of list items. With HashSets, it can perform the check instantly.

Go through your list of directory names alphabetically. Every time you look at a directory name, pass it through a method that will remove " - Copy..." etc from the end using substring methods. If that processed directory name is not part of the hash set, then add it. If it is already part of the hash set, delete it.

Are the OSU files identical (e.g., there are no minor differences like different white spacing)? If so, you can pass them through a checksum algorithm to compare them. Basically, a file will produce a hash string based on its contents. If another file has the exact same contents, it will produce an identical hash string. If you change just one character, the hash string will be completely different. It's a fast and efficient way to tell if files are identical.

To see if you've encountered a file before, you can add file hash strings to a hash set (sorry for the confusing terminology). If a hash string already exists in your set, you can know for sure that you've seen the file before.

Hope this helps.