2
\$\begingroup\$

Implementing Linux style file name wild cards on Windows 10 PowerShell

While implementing an objected oriented wc on Windows 10 in visual studio C++ I implemented a recursive file search with a weak attempt at pattern matching (regular expressions in file names). While unit testing it I discovered that it did not do a good job of file name matching. An example is that since the source code files of the wc program are a moving target, but I want to use those source files as test input for the wc program I created copies of the files with the file names ending in .cpp.txt and .h.txt. My wc program could not find these files separately, only as .txt files.

The code presented in this question fixes that bug in the program. This is only a stand alone test program. I haven't merged the code into the wc program yet.

This is my first attempt to use C++ regular expressions (std::regex).
One function in this code was contributed by another regular user here on Code Review, Toby Speight. I copied that code and used it as the basis for another function, the one where I actually implement a regex.

Questions:

  • Are any of the Linux wild card characters missing?
  • This should be pattern matching against file names only, does the regex in the code miss any legal characters in file names?
  • Can the use of std::regex be improved here?
  • Is there anything that isn't portable in this code?

Requirements

This code requires gcc12 to compile, the ranges and views implementation are not available in gcc11.
This code will compile in Visual Studio 2022 when C++20 is selected.

/*
 * Testing Linux Shell file regular expressions on Windows 10. 
 * Need to be able to parse file specifications such as *.cpp.txt as well as
 * *.cpp or *.txt.
 */

#include <filesystem>
#include <iostream>
#include <ranges>
#include <regex>
#include <string>
#include <vector>

static const std::string BASE_SEARCH_DIR = "C:\\Users\\PaulC\\Documents\\ProjectsNfwsi\\StaticCodeEvalTools\\wconsteroids";
static const std::string questionMarkReplacement("[a-zA-Z0-9:$@#-_./]");
static const std::string starReplacement(questionMarkReplacement + "*");

namespace fsys = std::filesystem;

static std::string convertWildCards(std::string testInput)
{
    std::string regexString;
    try
    {
        std::regex qmark("\\?");
        regexString = std::regex_replace(testInput, qmark, questionMarkReplacement);
        std::regex asterisk("\\*");
        regexString = std::regex_replace(regexString, asterisk, starReplacement);
    }
    catch (std::regex_error e)
    {
        std::cerr << "Regex Error: " << e.what() << "\n";
        exit(EXIT_FAILURE);
    }

    return regexString;
}

struct SubDirNode
{
    fsys::path fileSpec;
    bool discovered = false;
    bool searchedFiles = false;

    SubDirNode(fsys::path path)
        : fileSpec{ std::move(path) }
    {
    }

    bool operator==(const SubDirNode& other) const
    {
        return fileSpec == other.fileSpec;
    }
    bool operator!=(const SubDirNode& other) const = default;
};

static bool SearchSubDirs = true;
static std::vector<SubDirNode> subDirectories;

/*
 * Search the current directory for sub directories.
 */
static auto findSubDirs(const SubDirNode& currentDir)
{
    // The code and comments in this function was contributed by @tobyspeight
    auto is_missing = [](const SubDirNode& branch) {
        return std::ranges::find(subDirectories, branch) == subDirectories.end();
    };

    auto subdirs = fsys::directory_iterator{ currentDir.fileSpec }
    | std::views::filter([](auto& f) { return f.is_directory(); })
        | std::views::transform([](auto& f)->SubDirNode { return f.path(); })
        | std::views::filter(is_missing);

    // TODO (C++23?) return std::vector(subdirs);
    auto newSubDirs = std::vector<SubDirNode>{};
    std::ranges::copy(subdirs, std::back_inserter(newSubDirs));
    return newSubDirs;
}

static bool discoverSubDirs()
{
    bool discoveredPhaseCompleted = true;
    std::vector<SubDirNode> newSubDirs;

    for (std::size_t i = 0; i < subDirectories.size(); i++)
    {
        if (!subDirectories[i].discovered)
        {
            std::vector<SubDirNode> tempNewDirs = findSubDirs(subDirectories[i]);
            if (tempNewDirs.size())
            {
                discoveredPhaseCompleted = false;
                newSubDirs.insert(newSubDirs.end(), tempNewDirs.begin(),
                    tempNewDirs.end());
            }
            subDirectories[i].discovered = true;
        }
    }

    // We are done searching the current level, append the new sub directories
    // to the old.
    subDirectories.insert(subDirectories.end(), newSubDirs.begin(), newSubDirs.end());
    return discoveredPhaseCompleted;
}

static std::vector<std::string> fileList;
static auto searchDirectoryForFilesByPattern(SubDirNode currentDir,
    std::string partialFileSpec)
{
    auto is_missing = [](const std::string& branch) {
        return std::ranges::find(fileList, branch) == fileList.end();
    };

    std::vector<std::string> newFiles = {};
    std::string patternString = convertWildCards(partialFileSpec);
    try
    {
        std::string dir = currentDir.fileSpec.string();
        std::regex pattern(patternString);
        auto is_type = [pattern](auto f) {
            return std::regex_search(f.path().string(), pattern);
        };

        auto files = fsys::directory_iterator{ currentDir.fileSpec }
            | std::views::filter([](auto& f) { return f.is_regular_file() ||
                f.is_character_file(); })
            | std::views::filter(is_type)
            | std::views::transform([](auto& f) { return f.path().string(); })
            | std::views::filter(is_missing);

            std::ranges::copy(files, std::back_inserter(newFiles));
    }
    catch (std::regex_error& re)
    {
        std::cerr << "Regex Error: " << re.what() << "\n";
        std::cerr << patternString << "\n";
        exit(EXIT_FAILURE);
    }

    return newFiles;
}

static void searchAllDirectoriesForFiles(std::string partialFileSpec)
{
    for (auto currentDir : subDirectories)
    {
        if (currentDir.discovered && !currentDir.searchedFiles)
        {
            std::vector<std::string> newFiles =
                searchDirectoryForFilesByPattern(currentDir, partialFileSpec);
            std::copy(newFiles.begin(), newFiles.end(), std::back_inserter(fileList));
            currentDir.searchedFiles = true;
        }
    }
}

static void discoverAllSubDirs()
{
    bool discoveryPhaseCompleted = false;
    while (!discoveryPhaseCompleted)
    {
        discoveryPhaseCompleted = discoverSubDirs();
    }
}

static std::vector<std::string> findAllFilesMatching(std::string partialFileSpec)
{
    fileList.clear();
    std::filesystem::path cwd = BASE_SEARCH_DIR;
    if (subDirectories.empty())
    {
        SubDirNode root(cwd);
        root.discovered = (SearchSubDirs) ? false : true;
        subDirectories.push_back(root);
    }

    discoverAllSubDirs();
    searchAllDirectoriesForFiles(partialFileSpec);
    
    return fileList;
}

static void testFindAllFilesMAtching(std::string partialFileSpec)
{
    std::cout << "Searching for files that match " << partialFileSpec << " found:\n";

    std::vector<std::string> matchingFiles = findAllFilesMatching(partialFileSpec);
    for (auto fileName : matchingFiles)
    {
        std::cout << "\t" << fileName << "\n";
    }
    std::cout << "\n";
}

int main(int argc, char* argv[])
{
    testFindAllFilesMAtching("CmdLineFileExtractor.cpp");
    testFindAllFilesMAtching("CmdLineFileExtractor.*");
    testFindAllFilesMAtching("Cmd*.cpp");
    testFindAllFilesMAtching("Cmd*.*");
    testFindAllFilesMAtching("Cmd*.cpp.txt");
    testFindAllFilesMAtching("Cmd*.*");
    testFindAllFilesMAtching("*.h");
    testFindAllFilesMAtching("*.cpp");
    testFindAllFilesMAtching("*.h.txt");
    testFindAllFilesMAtching("*.cpp.txt");
}

You will find some of this code in the current version of wconsteroids.

\$\endgroup\$

1 Answer 1

1
\$\begingroup\$

? in a wildcard pattern matches any character, so we want . in place of [a-zA-Z0-9:$@#-_./]. Actually, it will match a newline, so probably (.|\n) would be better.

std::regex_error should not be thrown unless we made a programming error; I'd replace the contents of that catch with an assert(false) and abort().

The comment (written by me) // TODO (C++23?) return std::vector(subdirs); is inaccurate. The correct and idiomatic C++23 will use | std::ranges::to<std::vector>().

\$\endgroup\$
2
  • \$\begingroup\$ Since this is finding files I'm pretty sure newline is excluded. Can you have a newline in a file name? \$\endgroup\$
    – pacmaninbw
    Dec 21, 2022 at 22:27
  • \$\begingroup\$ Yes, of course. On a POSIX system, any byte can be part of a filename except for \0 and / (which is the directory separator). (Certain filesystem types, notably VFAT, restrict characters in a filename, but most do not). \$\endgroup\$ Dec 21, 2022 at 22:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.