5
\$\begingroup\$

I am an undergraduate CS student trying to implement simple unit test framework on C++ as a pet project.

The framework has an assertion macro like ASSERT_EQ(var1, var2), which checks whether two variables are equal or not. If the assertion fails, I want to print a failure message to std::cerr and provide as much information as possible about failed comparison. Therefore, values var1 and var2 are to be printed. But what if there is no corresponding operator << for the variables' type(s)? In this case I have to provide output operators for the types, so the framework is able to print the values in a "default" way. However, a default output operator should not be called for some type T if the type already has output operator.

With a relative success, I employed SFINAE for it in the following way.

file "sfinae_print.hpp":

#include <type_traits> // for enable_if
#include <iostream>
#include <utility>

//////////////////////////////////////
//   META TYPE CHECK: OPERATOR <<   //
//////////////////////////////////////

// checks whether T has declared output iterator or not
template<typename T>
class has_output_operator
{
private:

    // decltype(...) statically evaluates the type of passed expression
    //      fail leads to substitution failure in the context
    template<typename U, typename = decltype(std::cout << std::declval<U>())>
    static constexpr bool
    check(nullptr_t) noexcept
    {
        return true;
    }

    // less specialized function - check(nullptr_t) is
    //      more preferable (but may cause substitution failure)
    template<typename ...>
    static constexpr bool check(...) noexcept
    {
        return false;
    }

public:
    static constexpr bool value{check<T>(nullptr)};
};

///////////////////////////////////////
//   META TYPE CHECK: IS ITERATABLE  //
///////////////////////////////////////

// checks whether T is iteratable (supports begin() and end()) or not
template<typename T>
class is_iteratable
{
private:

    template<typename U>
    static constexpr decltype(std::begin(std::declval<U>()),
            std::end(std::declval<U>()),
            bool())
    check(nullptr_t) noexcept
    {
        return true;
    }

    template<typename ...>
    static constexpr bool check(...) noexcept
    {
        return false;
    }

public:
    static constexpr bool value{check<T>(nullptr)};
};

template<typename T>
void print_meta_info(std::ostream& os = std::cout)
{
    os << "is iteratable: " << is_iteratable<T>::value << std::endl;
    os << "has output operator: " << has_output_operator<T>::value << std::endl;
    os << std::endl;
}

/////////////////////////
//   ITERATABLE TYPE   //
/////////////////////////

// "default" output operators:

// operator << for iteratable type with no other output operators
template <typename T>
typename std::enable_if<is_iteratable<T>::value &&
                        !has_output_operator<T>::value,
        std::ostream&>::type
operator << (std::ostream& os, const T& obj)
{
    bool flag{false};

    os << "{";
    for(const auto& unit : obj)
    {
        if(flag)
        {
            os << ", ";
        }
        flag = true;
        os << unit;
    }
    os << "}";

    return os;
}

//////////////
//   PAIR   //
//////////////
// same for a pair:
template <typename LHS, typename RHS>
typename std::enable_if<!has_output_operator<std::pair<LHS, RHS>>::value, std::ostream&>::type
operator << (std::ostream& os, const std::pair<LHS, RHS>& obj)
{
    return os << "{" << obj.first << "," << obj.second << "}";
}

/////////////////////////////
//   NON-ITERATABLE TYPE   //
/////////////////////////////
// same for non-iteratable type:
template <typename T>
typename std::enable_if<!is_iteratable<T>::value &&
                        !has_output_operator<T>::value,
        std::ostream&>::type
operator << (std::ostream& os, const T& value)
{
    // cannot be printed
    // some failure is bound to occur
    return os << value;
    // print POD with reflection?
}

Usage example:

#include <iostream>
#include <vector>
#include <string>
#include <map>
#include <set>

#include "sfinae_print.hpp"

std::ostream& operator << (std::ostream& os, const std::set<int> obj)
{
    return os << "explicitly defined operator << for set<int>";
}

int main()
{
    // printing values using "default" output operators: 
    const std::vector<std::string> v_str{"sfinae", "is", "dope"};
    std::cout << v_str << std::endl;

    const std::vector<int> v_int{1, 2, 3};
    std::cout << v_int << std::endl;

    const std::map<int, int> map_int_int{{1, 2}, {3, 4}};
    std::cout << map_int_int << std::endl;

    // std::set<int> has defined operator <<, thus there is no need in default operator << 
    const std::set<int> set_int{1, 9, 1, 7};
    std::cout << set_int << std::endl;

    return 0;
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.17)

# set the project name and version
project(SFINAE VERSION 0.1)

# specify the C++ standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)

# set a variable for .cpp source files
set(SOURCES
        main.cpp
)

# set a variable .h headers
set(HEADERS
        sfinae_print.hpp
)

# add the executable
add_executable(SFINAE ${SOURCES} ${HEADERS})

# enable all warnings during compile process
# append flag to previously defined flag
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall  -Wextra")

Is it a tolerable solution? Could you please give me any suggestions on how to improve it?

\$\endgroup\$

1 Answer 1

4
\$\begingroup\$

I like the idea generally—it’s not bad at all—but the one thing that really raises my hackles about it is that you’re using operator<<(ostream&, T), the standard stream inserter interface, as your basis testing output function.

The reason it bothers me is three-fold. First, what you’re doing by doing this is changing the public interface of the type under test (assuming the type doesn’t already have a stream inserter function defined, of course—if it does, then no harm no foul). Now, this is somewhat unavoidable if you actually want to print the value of the type for useful test output… but… operator<<(ostream&, T) is a very generic interface component. It’s the standard stream interface. It’s one thing to add a very esoteric, your-test-framework-only function to the interface… that’s not ideal, but, as I said, it’s unavoidable. But adding an interface component as universal as operator<<(ostream&, T)… that seems a bit reckless.

Second, I may have very good reasons for not wanting operator<<(ostream&, T) to be defined for some T. In fact, I may even have done operator<<(ostream&, T) = delete. What happens then?

And third, even if I already have a stream inserter defined for my type… I may actually want different output for testing than the “normal” output. For example, I may want to print hidden flags or data in the type that normally wouldn’t want exposed in normal use.

Another issue is that by adding stream inserter functions for things like std::pair and literally any iterable type, you have radically changed the environment. Again, this is unavoidable—any test framework, by axiomatic necessity, must exist in the test environment when it wouldn’t exist in normal operation. But again, it’s one thing for your test framework to add stuff that by-and-large won’t interact with anything else unless specifically made to (for example, by putting it in its own namespace that won’t conflict with other namespaces)… it’s quite another to add stuff that will likely—or in this case, almost certainly—get entangled with the the operations of the system-under-test.

For example, my type may be designed to check for a stream inserter for pair<X, Y>, which the user could provide if they want custom behaviour, and if one doesn’t exist, do a default output format instead. Since your test framework adds a stream inserter for all pair<X, Y> (that don’t already have one), that means my type will never have to use its default output format. That means my type behaves differently depending on whether it’s being tested or not… which is absolutely not what you want.

Yet another issue—one that you may already have run into—is the problem of where to put all your fallback stream inserter machinery. There are two arguments to standard stream inserters: one is in the std namespace (namely std::ostream), and the other is in an arbitrary namespace (namely, the namespace of T). You can’t put your fallback inserters in your test framework namespace (which you should absolutely have), because then they won’t be found for arbitrary types. You would either have to put it in the std namespace (no, don’t do that), or the namespace of every T you might possibly use (which is impossible), or—as you’ve done—in the “working” namespace (which in your case is the global namespace). Bad solutions all around.

Okay, so enough doom and gloom: I’ve explained the problem, now what’s the solution?

The solution is simply to not use the standard stream inserter interface as your test framework’s go-to output interface. Instead, use something that’s very loudly specific only to your test framework.

You have basically two options.

Option 1 is to use a custom function that can be found by ADL. This is what Boost.Test does. All you need to do is define a function in your type’s namespace with the following signature:

auto boost_test_print_type(std::ostream&, T const&) -> std::ostream&;

For any variable t, Boost.Test basically does:

  1. Try an ADL call to boost_test_print_type(cout, t).
  2. If that fails, try cout << t.
  3. If that fails, error.

(There’s actually much more to it, including supporting the BOOST_TEST_DONT_PRINT_LOG_VALUE macro. But this is the gist of it.)

So if you don’t want to or can’t provide a stream inserter for your type… OR you want different output for testing versus “normal” output for your type, you can just add a function your_test_framework_printer_function(ostream&, T) in T’s namespace (so ADL can find it). And of course you can have your_test_framework_printer_function(ostream&, pair<X, Y>) or your_test_framework_printer_function(ostream&, AnyIterableType) as defaults within your framework if you want.

The key thing is that long, loud name is highly unlikely to conflict with anything else.

Option 2 is to create a custom stream type. It doesn’t need to be complex—it could just be a simple wrapper around std::ostream. Then for each type you want special test output for, you’d define a function like:

namespace my_ns {

struct my_type {};

// You probably wouldn't put this in the header file, you'd probably define
// it only in the actual unit test file. But, whatever:
auto operator<<(test_framework::ostream& o, my_type const& v)
{
    // Do whatever you want here.
    o << "[my_type]";

    return o;
}

} // namespace my_ns

Your test framework, using a test_framework::ostream internally (which could just be a thin wrapper around a std::ostream), would then do:

  1. Try o << t (with o being a test_framework::ostream).
  2. If that fails, try o.wrapped_stream << t (where the wrapped stream is a std::ostream).
  3. If that fails, error.

And again, you can provide defaults for std::pair, iterables, and whatever you like.

The key thing here is that that special stream type, test_framework::ostream, is specific to the test framework, and thus won’t interact/interfere with the environment or type-under-test.

In summary, what you’ve done so far is correct… but instead of working around the standard interface incantation operator<<(std::ostream&, T), you should use a custom interface incantation that won’t interact or get entangled with the environment or the type-under-test. There are several ways to do that (I just mentioned 2), and you can still use operator<<(std::ostream&, T) as a default.

Okay, now onto the code itself:

// checks whether T has declared output iterator or not
template<typename T>
class has_output_operator
{
private:

    // decltype(...) statically evaluates the type of passed expression
    //      fail leads to substitution failure in the context
    template<typename U, typename = decltype(std::cout << std::declval<U>())>
    static constexpr bool
    check(nullptr_t) noexcept
    {
        return true;
    }

    // less specialized function - check(nullptr_t) is
    //      more preferable (but may cause substitution failure)
    template<typename ...>
    static constexpr bool check(...) noexcept
    {
        return false;
    }

public:
    static constexpr bool value{check<T>(nullptr)};
};

This isn’t “wrong”… but this really isn’t the way to do type traits in C++17 and beyond. Hell, it isn’t even the way to do type traits in C++11 anymore (because you can backport void_t pretty trivially).

The tool you need is std::void_t. Trust me, it’s magical. This is what the type trait above looks like using void_t:

template <typename T, typename = void>
struct has_output_operator : std::false_type {};

template <typename T>
struct has_output_operator<T, std::void_t<decltype(std::cout << std::declval<T>())>> : std::true_type {};

That’s it. Basically 4 lines of code, 3 of which are ENTIRELY boilerplate, and the 4th is mostly boilerplate. And once you know the magic, you know exactly where to look for the meat of the type trait (which is just the stuff within void_t’s angle brackets). No need for comments explaining function overload precedence or any of that stuff.

By the by, rather than:

std::cout << std::declval<T>()

as the test, I’d suggest:

std::declval<std::ostream&>() << std::declval<T>()

because it’s technically possible to have different behaviour when doing output to cout rather than a generic ostream. (I mean, any type that does this is evil, but….) As a bonus, now you can just include <iosfwd> rather than the much heavier <iostream>. (Well, except you need it for cout later.)

As for the is_iterable type trait, the same void_t trick applies. In fact, the cppreference example for void_t has is_iterable.

// same for non-iteratable type:
template <typename T>
typename std::enable_if<!is_iteratable<T>::value &&
                        !has_output_operator<T>::value,
        std::ostream&>::type
operator << (std::ostream& os, const T& value)
{
    // cannot be printed
    // some failure is bound to occur
    return os << value;
    // print POD with reflection?
}

I may be misunderstanding what’s going on here, but if I’m reading this correctly, you want this function to be available for a T when os << value fails to compile? (And it’s not a type that supports iteration, which you provide a separate overload for.) But… if that’s the case, then you already know the body won’t compile.

Except… it will compile… because you just wrote that missing inserter function. Thanks to the above function, os << value will now compile… and call os << value… which calls os << value… which calls os << value….

See for yourself. In your main.cpp, add two lines: struct foo {}; just before main(), and then std::cout << foo{}; anywhere in main(). Compile, run, watch the fireworks. (To get a better visual sense of what’s going on, you could add a line in that function above that prints something like"hi!\n" or maybe a counter before os << value.)

So… what was supposed to happen here? I mean, you could fix the recursion by not trying to stream out value. But… then what? The comment indicates you might want to use reflection to print the type’s value? I guess lacking reflection, you could at least print the type’s name using something like abi::__cxa_demangle or maybe picking apart __PRETTY_FUNCTION__.

Summary

I’d strongly recommend against working with the standard stream inserter interface. When you do, you’re screwing around with the environment and the widely-used public interface of the type-under-test, which could break a lot of stuff, and even if it doesn’t, it will probably make things behave differently during testing versus regular operation (which is a HUGE no-no). Instead, make a custom output interface—either a function or a custom stream—for your test framework; one that is clearly distinct and won’t get mixed up with the usual environment of the system-under-test.

Use std::void_t to make your type traits. It’s a thousand times easier, and much easier to read, understand, and maintain.

Beware that infinite recursion bug in your fallback inserter.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.