C++ Mock Library: Part 1

Question

Parts

• C++ Mock Library: Part 1
• C++ Mock Library: Part 2
• C++ Mock Library: Part 3
• C++ Mock Library: Part 4
• C++ Mock Library: Part 5
• C++ Mock Library: Part 6

Note:

If you see an extra T on a macro ignore for now.
I will get to this in a subsequent review. There is to much for one review so I will do in stages.

Example: MOCK_FUNC() and MOCK_TFUNC() for now pretend these are the same. I just don't want to have to edit my code before pasting here. I will go over these oddities in a later review.

Motivation

So writing unit tests for my code!

I use Google Test as the basic framework. But my unit tests files were 3947+ lines long (the code was only about 500 lines). A lot of this is repeated code and looked very ugly in the tests. So I though lets try and improve the framework a bit to reduce the repetitive nature of the unit tests.

So this is the code for my mock framework (reduced the unit test files to 1713 and does more testing).

Interface

OK. So this is the part I like least. But I could not think of a better way.

Any function that you want to mock must be wrapped in MOCK_FUNC(). In debug and release mode this simply does nothing and the function is just used like normal. In my code coverage build (which is where the unit tests are run). These functions are swapped out by a functor object.

Example:

Socket::Socket(std::string const& hostname, int port, Blocking blocking)
    : fd(-1)
{
    fd = MOCK_FUNC(socket)(PF_INET, SOCK_STREAM, 0);
    if (fd == -1) {
       // STUFF removed
    }

    // STUFF removed

    HostEnt* serv = nullptr;
    while (true)
    {
        serv = MOCK_FUNC(gethostbyname)(hostname.c_str());
        if (serv == nullptr && h_errno == TRY_AGAIN) {
            continue;
        }
        // STUFF removed
    }

    // STUFF removed

    if (MOCK_FUNC(connect)(fd, reinterpret_cast<SocketAddr*>(&serverAddr), sizeof(serverAddr)) != 0)
    {
        // STUFF removed
    }
    if (blocking == Blocking::No)
    {
        if (MOCK_TFUNC(fcntl)(fd, F_SETFL, O_NONBLOCK) == -1)
        {
            // STUFF removed
        }
    }
}

You will see several mocked functions: MOCK_FUNC(socket), MOCK_FUNC(gethostbyname), MOCK_FUNC(connect), MOCK_TFUNC(fcntl).

So my build tools (make) will supply the macro -DMOCK_FUNC\(x\)=::x for debug and release builds, while for coverage builds it will add --include coverage/MockHeaders.h so that all files include this file automatically and have some more complex definitions (coming below).

The idea here is there should be zero cost in release or debug builds. But for coverage builds we get some extra functionality.

Generated files:

It generates two other files `coverage/MockHeaders.h` and `coverage/MockHeaders.cpp` these files contain the majority of the code to review. Its a lot so I will split it over a couple of reviews. These files simply need to part of the build.

The important part is that for every mocked out functions it generates a global functor variable.


namespace ThorsAnvil::BuildTools::Mock
{
// Functor
// For each mocked out function we have a functor.
// This can hold an alternative version of the function, but by default it is
// initialized by the variable it is supposed to override.
// So if you want to use the normal function during the unit tests you don't
// need to do anything and you get the normal function being called.
// Alternatively there are objects provide to override this function with a lambda.
//
MockFunctionHolder<RemoveNoExcept<ThorsAnvil::BuildTools::Mock::FuncType_fcntl>> MOCK_BUILD_MOCK_SNAME(fcntl)("fcntl", ::fcntl);
MockFunctionHolder<RemoveNoExcept<ThorsAnvil::BuildTools::Mock::FuncType_open>> MOCK_BUILD_MOCK_SNAME(open)("open", ::open);
MockFunctionHolder<RemoveNoExcept<decltype(::close)>> MOCK_BUILD_MOCK_SNAME(close)("close", ::close);
MockFunctionHolder<RemoveNoExcept<decltype(::connect)>> MOCK_BUILD_MOCK_SNAME(connect)("connect", ::connect);
MockFunctionHolder<RemoveNoExcept<decltype(::gethostbyname)>> MOCK_BUILD_MOCK_SNAME(gethostbyname)("gethostbyname", ::gethostbyname);
MockFunctionHolder<RemoveNoExcept<decltype(::pipe)>> MOCK_BUILD_MOCK_SNAME(pipe)("pipe", ::pipe);
MockFunctionHolder<RemoveNoExcept<decltype(::read)>> MOCK_BUILD_MOCK_SNAME(read)("read", ::read);
MockFunctionHolder<RemoveNoExcept<decltype(::shutdown)>> MOCK_BUILD_MOCK_SNAME(shutdown)("shutdown", ::shutdown);
MockFunctionHolder<RemoveNoExcept<decltype(::socket)>> MOCK_BUILD_MOCK_SNAME(socket)("socket", ::socket);
MockFunctionHolder<RemoveNoExcept<decltype(::write)>> MOCK_BUILD_MOCK_SNAME(write)("write", ::write);

Code Review

OK. Finally to the part where you get to look at some code:

Helper Macros:

// Macros to expand a function name into the name of a global functor object.
// MOCK_FUNC used in user code includes the full namespace prefix.
// MOCK_BUILD_MOCK_SNAME used to name the object inside a namespace scope.
//
#define MOCK_BUILD_MOCK_NAME_EXPAND_(name)      mock_ ## name
#define MOCK_BUILD_MOCK_NAME_EXPAND(name)       MOCK_BUILD_MOCK_NAME_EXPAND_(name)

#define MOCK_BUILD_MOCK_NAME(name)                          ThorsAnvil::BuildTools::Mock:: MOCK_BUILD_MOCK_NAME_EXPAND(name)
#define MOCK_BUILD_MOCK_SNAME(name)             MOCK_BUILD_MOCK_NAME_EXPAND(name)
#define MOCK_FUNC(name)                         MOCK_BUILD_MOCK_NAME(name)

Helper Classes

namespace ThorsAnvil::BuildTools
{
    template <typename T>
    struct RemoveNoExceptExtractor
    {
        using Type = T;
    };

    template <typename R, typename... Args>
    struct RemoveNoExceptExtractor<R(Args...) noexcept>
    {
        using Type = R(Args...);
    };

    template <typename F>
    using RemoveNoExcept = typename RemoveNoExceptExtractor<F>::Type;
}

Functor Object

namespace ThorsAnvil::BuildTools::Mock
{

    template<typename Func>
    struct MockFunctionHolder
    {
        using Param   = ParameterType<Func>;

        std::string             name;
        std::function<Func>     action;

        template<typename F>
        MockFunctionHolder(std::string const& name, F&& action);
        std::string const& getName() const;
        template<typename... Args>
        ReturnType<Func> operator()(Args&&... args);
    };


    // -------------------------
    // MockFunctionHolder
    // -------------------------
    
    template<typename Func>
    template<typename F>
    MockFunctionHolder<Func>::MockFunctionHolder(std::string const& name, F&& action)
        : name(name)
        , action(std::move(action))
    {}
    
    template<typename Func>
    std::string const& MockFunctionHolder<Func>::getName() const
    {
        return name;
    }
    
    template<typename Func>
    template<typename... Args>
    ReturnType<Func> MockFunctionHolder<Func>::operator()(Args&&... args)
    {
        return action(std::forward<Args>(args)...);
    }

}

Writing a unit test mocking out a function

When writing a unit test to simply mock out a function you use the macro MOCK_SYS() that replaces a function with a macro:

TEST(UniTestBloc, MyUnitTest)
{
    MOCK_SYS(socket,        [](int, int, int)                      {return 12;});
    MOCK_SYS(connect,       [](int, sockaddr const*, unsigned int) {return 0;});
    MOCK_SYS(gethostbyname, [](char const*) {
        static char* addrList[] = {""};
        static hostent result {.h_length=1, .h_addr_list=addrList};
        return &result;
    });

    auto action = [](){
        ThorsSocket::Socket socket("www.google.com", 80);
        // Do tests
    };
    ASSERT_NO_THROW(action());
}

The definition of MOCK_SYS is:

#define MOCK_SYS_EXPAND_(type, func, mocked, lambda)        ThorsAnvil::BuildTools::Mock::MockOutFunction<type>  mockOutFunction_ ## func(mocked, lambda)
#define MOCK_SYS_EXPAND(type, func, mocked, lambda)         MOCK_SYS_EXPAND_(type, func, mocked, lambda)
#define MOCK_SYS(func, lambda)                              MOCK_SYS_EXPAND(ThorsAnvil::BuildTools::RemoveNoExcept<decltype(::func)>, func, MOCK_BUILD_MOCK_NAME(func), lambda)

And the object it creates is:

namespace ThorsAnvil::BuildTools::Mock
{
    template<typename Func>
    struct MockOutFunction
    {
        std::function<Func>         old;
        MockFunctionHolder<Func>&   orig;
        MockOutFunction(MockFunctionHolder<Func>& orig, std::function<Func>&& mock);
        ~MockOutFunction();
    };

    // -------------------------
    // MockOutFunction
    // -------------------------
    
    template<typename Func>
    MockOutFunction<Func>::MockOutFunction(MockFunctionHolder<Func>& orig, std::function<Func>&& mock)
        : old(std::move(mock))
        , orig(orig)
    {
        swap(old, orig.action);
    }
    
    template<typename Func>
    MockOutFunction<Func>::~MockOutFunction()
    {
        swap(old, orig.action);
    }
}

Part 1 done

OK. Thats the basics. This part simply is about haveing a function that can be repalced by a lambda during a UNIT TEST. This by itself is not earth shattering and does not save much code. The next question will expand on this.

Answer 1

Macros are difficult

Your code relies on macros, and macros are notoriously difficult to work with if you go beyond the simple #define CONSTANT and #ifdef FLAG. Let's just look at what happens if you don't enable mocking:

So my build tools (make) will supply the macro -DMOCK_FUNC\(x\)=::x for debug and release builds, […]

I immediately see two problems with this. The first is that this will only work correctly for functions in the global namespace. If you would write:

using std::format;
auto text = MOCK_FUNC(format)("Hello, {}!", "world");

Then this would fail in a release build. And the following would work in a release build, but would break a coverage build:

auto text = MOCK_FUNC(std::format)("Hello, {}!", "world");

The second issue is that not everything that looks like a function is actually a function. Consider this:

FD_SET fd_set;
MOCK_FUNC(FD_CLR)(&fd_set);

FD_CLR() expands to something hygienic like do {…} while(0), but of course then ::FD_CLR() will not work correctly. While this is a rather obvious example, there might be more innocent looking functions that are actually implemented as macros, which also might depend on the platform you are on.

So what to do about this? First, just use -DMOCK_FUNC(x)=x to fix release builds. Second, if you want to handle namespaces, perhaps the only way is to stringify the name of the functions, and use it as a lookup into a dictionary:

#define MOCK_FUNC(func) MOCK_LOOKUP_FUNC(#func)

Or perhaps even MOCK_LOOKUP_FUNC<#func> since C++20. With some effort, most if not all of the hashing can be done at compile-time. Since you are autogenerating files anyway, you can use perfect hashing (with a tool like gperf).

Alternatives

What's great about your code is that you can just write MOCK_SYS(function, replacement) in your tests. It's less great to have to find all instances of function in your code base and to wrap it in MOCK_FUNC(). The latter is very bad for maintenance: it's easy to forget to do this when making changes to your actual code. I would recommend finding a way to make it work without having to use MOCK_FUNC() at all.

You have a preprocessing step anyway that finds MOCK_FUNC() calls and uses it to generate coverage/MockHeaders.*. Instead you can have that step look for MOCK_SYS() calls in your tests, and then generate code to replace the standard library functions with custom ones, like suggested in this StackOverflow answer. Other possibilities are using the compiler to wrap functions, just overriding library functions (but then you need some tricks to be able to call the original), and perhaps there are other possibilities as well.