# A tuple unpacking macro that approximates Python elegance

## Motivation

Although I love coding in C++, I sometimes yearn for the syntactic sugar of Python. C++11 has somewhat eased the pain by such beautiful analogies like this:

# python
for i in [1, 5, 7]:

// C++11
for(auto i: {1, 5, 7})


Unfortunately, C++ puts an end to this as soon as you want to do such crazy things as having the element index in a range based for.

Where Python gives you

for index, element in enumerate(elements):


C++ leaves us with writing a "normal" loop like

for(int index = 0; index < elements.size(); ++i)
//do something with elements[index]


(which is slow with non random access containers) or with uglier loops that involve iterators. Fortunately, it is possible to translate the beauty of Python and write a range adaptor that returns an (index, element&) tuple (or pair) and could be created and used like this:

for(auto indexElementPair: enumerate(elements))


Yet, this is only half the beauty of Python since we have to deal with this ugly pair using it like indexElementPair.first which is not the nicest form.

## Usage

This is where my code comes into play. I wanted to emulate the behavior of Python as close as possible. Although it is not possible to declare multiple (auto) variables in the range based for header (and unpack the tuple into them) we can do so right after it:

for(auto indexElementPair: enumerate(elements)) {
DECLARE_AND_TIE((index, element), indexElementPair);
// use index and element like in the python code
}


Note that the index and element variables are both declared as references and their type is automatically deduced from the corresponding tuple element.

The code is on github and in the following files:

## declare_and_tie.hpp

#ifndef _guard_DECLARE_AND_TIE_HPP_
#define _guard_DECLARE_AND_TIE_HPP_

#error "Boost preprocessor variadics support needed for DECLARE_AND_TIE!"
#endif

#include <boost/preprocessor/tuple/size.hpp>
#include <boost/preprocessor/tuple/to_seq.hpp>
#include <boost/preprocessor/seq/for_each_i.hpp>
#include <boost/preprocessor/if.hpp>
#include <boost/preprocessor/comparison/equal.hpp>
#include <boost/preprocessor/expand.hpp>
#include <boost/mpl/aux_/preprocessor/is_seq.hpp>
#include "is_boost_pp_tuple.hpp"
#include "require_trailing_semicolon.hpp"

/**
* @brief Declare references to tuple elements with the given names
* @param UNPACKED_ELEMENTS a boost preprocessor tuple of names for references
* @param TUPLE to unpack
*
* Example usages:
*
* std::tuple<int, float, string> cppTuple;
* DECLARE_AND_TIE((a, b, c), cppTuple);
*
* std::tuple<int, float, string> cppTuple;
* //ignoring the float element
* DECLARE_AND_TIE((a, _, c), cppTuple);
*/
#define DECLARE_AND_TIE(UNPACKED_ELEMENTS, TUPLE)                              \
static_assert(BOOST_PP_IS_TUPLE(UNPACKED_ELEMENTS),                        \
"First parameter must be a tuple!");                         \
static_assert(                                                             \
!COMPARE_PP_TUPLE_TO_CPP_TUPLE_SIZE(UNPACKED_ELEMENTS, >, TUPLE),      \
"Too many unpacked elements for tuple");                               \
static_assert(                                                             \
!COMPARE_PP_TUPLE_TO_CPP_TUPLE_SIZE(UNPACKED_ELEMENTS, <, TUPLE),      \
"Too few unpacked elements for tuple");                                \
_impl_DECLARE_AND_TIE_VARIABLES(UNPACKED_ELEMENTS, TUPLE);                 \
REQUIRE_TRAILING_SEMICOLON()

#define COMPARE_PP_TUPLE_TO_CPP_TUPLE_SIZE(UNPACKED, REL, TUPLE)               \
(BOOST_PP_TUPLE_SIZE(UNPACKED) REL std::tuple_size<decltype(TUPLE)>::value)

#define _impl_DECLARE_AND_TIE_ONE_VARIABLE(r, TUPLE, INDEX, VARNAME)           \
BOOST_PP_IF(_impl_TOKEN_IS_UNDERSCORE(VARNAME),                            \
_impl_DECLARE_AND_TIE_UNDERSCORE,                              \
_impl_DECLARE_AND_TIE_VARIABLE)(r, TUPLE, INDEX, VARNAME)

#define _impl_TOKEN_IS_UNDERSCORE_TEST_(x) x

#define _impl_TOKEN_IS_UNDERSCORE(a)                                           \
BOOST_MPL_PP_IS_SEQ(                                                       \
BOOST_PP_CAT(_impl_TOKEN_IS_UNDERSCORE_TEST, a)((unused)))

#define _impl_DECLARE_AND_TIE_UNDERSCORE(r, TUPLE, INDEX, VARNAME)

#define _impl_DECLARE_AND_TIE_VARIABLE(r, TUPLE, ELEMENT_INDEX, VARIABLE_NAME) \
auto& VARIABLE_NAME = std::get<ELEMENT_INDEX>(TUPLE);

#define _impl_DECLARE_AND_TIE_VARIABLES(UNPACKED_ELEMENTS, TUPLE)              \
BOOST_PP_SEQ_FOR_EACH_I(_impl_DECLARE_AND_TIE_ONE_VARIABLE, TUPLE,         \
BOOST_PP_TUPLE_TO_SEQ(UNPACKED_ELEMENTS))

#endif /* _guard_DECLARE_AND_TIE_HPP_ */


## require_trailing_semicolon.hpp

#ifndef REQUIRE_TRAILING_SEMICOLON_HPP_
#define REQUIRE_TRAILING_SEMICOLON_HPP_

#include <boost/preprocessor/cat.hpp>

#define REQUIRE_TRAILING_SEMICOLON()                                           \
struct BOOST_PP_CAT(trailing_semicolon_required_on_line_, __LINE__) {}

#endif /* REQUIRE_TRAILING_SEMICOLON_HPP_ */


## unit_tests.cpp

#include "test.hpp"

#include <declare_and_tie.hpp>

BOOST_AUTO_TEST_CASE(Test_reference_to_tuple_elements) {
auto tuple = std::make_tuple(1, 0.5, std::string("hello world"));
DECLARE_AND_TIE((a, b, c), tuple);
BOOST_REQUIRE(std::is_reference<decltype(a)>::value);
BOOST_REQUIRE_EQUAL(&a, &std::get<0>(tuple));
BOOST_REQUIRE(std::is_reference<decltype(b)>::value);
BOOST_REQUIRE_EQUAL(&b, &std::get<1>(tuple));
BOOST_REQUIRE(std::is_reference<decltype(c)>::value);
BOOST_REQUIRE_EQUAL(&c, &std::get<2>(tuple));
}

template <class T>
using isConstRef = std::is_const<typename std::remove_reference<T>::type>;

BOOST_AUTO_TEST_CASE(Test_retain_tuple_constness) {
auto const tuple = std::make_tuple(1, 0.5, std::string("hello world"));

BOOST_REQUIRE(std::is_const<decltype(tuple)>::value);

DECLARE_AND_TIE((a, b, c), tuple);

BOOST_REQUIRE(isConstRef<decltype(a)>::value);
BOOST_REQUIRE(isConstRef<decltype(b)>::value);
BOOST_REQUIRE(isConstRef<decltype(c)>::value);
}

BOOST_AUTO_TEST_CASE(Test_retain_tuple_element_constness) {
std::tuple<const int, double, const char*> tuple(1, 0.5, "hello world");

BOOST_REQUIRE(!std::is_const<decltype(tuple)>::value);

DECLARE_AND_TIE((a, b, c), tuple);

BOOST_REQUIRE(isConstRef<decltype(a)>::value);
BOOST_REQUIRE(!isConstRef<decltype(b)>::value);
// the pointer is not const!
BOOST_REQUIRE(!isConstRef<decltype(c)>::value);
}

BOOST_AUTO_TEST_CASE(Test_multiple_ties) {
const auto tuple = std::make_tuple(1, 0.5, std::string("hello world"));
DECLARE_AND_TIE((a, b, c), tuple);
DECLARE_AND_TIE((d, e, f), tuple);
}

/*
BOOST_AUTO_TEST_CASE(Test_error_on_rvalue_cpp_tuple) {
DECLARE_AND_TIE((integer, real, string),
std::make_tuple(1, 0.5, std::string("hello world")));
}
*/

/*
BOOST_AUTO_TEST_CASE(Test_error_on_non_tuple_unpack_list) {
auto tuple = std::make_tuple(1, 0.5, std::string("hello world"));
DECLARE_AND_TIE(one_parameter, tuple);
}
*/

/*
BOOST_AUTO_TEST_CASE(Test_error_on_missing_trailing_semicolon) {
auto tuple = std::make_tuple(1, 0.5, std::string("hello world"));
DECLARE_AND_TIE((a, b, c), tuple)
}
*/

/*
BOOST_AUTO_TEST_CASE(Test_error_on_too_many_unpack_elements) {
std::tuple<const int, double, std::string> tuple(1, 0.5, "hello world");
DECLARE_AND_TIE((a, b, c, d), tuple);
}
*/

/*
BOOST_AUTO_TEST_CASE(Test_error_on_too_few_unpack_elements) {
std::tuple<const int, double, std::string> tuple(1, 0.5, "hello world");
DECLARE_AND_TIE((a, b), tuple);
}
*/


I omitted the code for is_boost_pp_tuple.hpp because it is not mine and it is only included because I could not find it in the boost sources (although there is a commit for it on some non existing SVN branch).

Obviously many of the unit tests are not compilable. Instead they should poster-child common usage errors and the expected error messages. If you have any idea of how to turn these dead tests into living ones I am very keen to hear about them.

## Features

• automatic type deduction (includes constness)
• sane error messages on wrong usage (at least on gcc):
• too many/few unpack elements
• first argument not a Boost.Preprocessor tuple
• second argument not a C++ tuple
• second argument is a temporary (or rather rvalue)
• no runtime cost for ignoring elements via _ (use like DECLARE_AND_TIE((a, _, _), tuple) to ignore all but the first element)

## Review goals

• correctness (and viability) of the approach including
• dangling reference/pointer problems
• exception safety
• concurrency problems
• (all) features supported on your platform of choice (I cannot test all compilers and operating systems, note that I require C++11 compliancy)
• usability issues (especially unanticipated error case that give unhelpful error messages)
• performance issues

In summary I want to know of any better way to achieve this.

I would also like to hear your opinion whether the _ placeholders are sensible. They complicate the design of DECLARE_AND_TIE and introduce a dependency on BOOST_MPL_PP_IS_SEQ from boost/mpl/aux_/preprocessor/is_seq.hpp which does not look like a stable include path.

In all use cases that I could think of they (the _ placeholders) could have been prevented by not having the ignored elements in the tuple in the first place.

## Known "error"

One situation where there is no sensible error message is the case of providing an empty preprocessor tuple for a one element C++ tuple:

std::tuple<int> tuple;
DECLARE_AND_TIE((), tuple);


This will dodge the size test because an empty preprocessor tuple has the size 1. There will be an error message because the variable that gets declared has no name.

I don't think it is very important to correct this because using a one element tuple is very unlikely in the first place and giving an empty unpack list should be obviously wrong. However, if you can show me that this problem is bigger than I thought or have a very easy workaround I would like to know.

The same problem (empty PP tuples having size 1) leads to this code giving a spurious error:

DECLARE_AND_TIE((), empty_tuple);


The size check will determine that the PP tuple has size 1 whereas empty_tuplehas size 0. However, I don't think that empty tuples should be a use case (but again I would like to hear if I am wrong on this).

## More motivation

Some of the arguments are along the line that this too much overhead for two lines of reference declarations. My initial motivating example is only a small subset. The technique should be applicable to macros of any size. Take for example this hypothetical code:

for(auto tuple: zip(numbers, names, ages, addresses))  {
// ...
}


You might find it acceptable to write instead

for(auto tuple: zip(numbers, names, ages, addresses))  {
auto &number = std::get<0>(tuple);
auto &name = std::get<1>(tuple);
auto &age = std::get<2>(tuple);
}


This is a lot of boilerplate code that allows for errors like wrong type declarations and incorrect indices (which is very likely due to copy&paste coding).

• Holy mother of macro jesus! Generally I disapprove of using macros to this extent. They are hard to debug, difficult to remember and generally a pain in the ass except for the person who wrote them. Then again that's only my opinion, I know there are hordes of macro gurus out there who will disagree with me. So take this as an opinion. – Emily L. Aug 29 '14 at 20:03
• Funny that you posted this now, as I was thinking about this exact problem yesterday. – Yuushi Aug 30 '14 at 5:53
• @EmilyL.: I would have liked to not use macros for this but I did not see any way around them. I tried to make the macro as user friendly as possible so if you don't want to wade through the implementation I'd still like to hear usage remarks. – Nobody Aug 30 '14 at 7:30
• @Yuushi: Rest assured that I have been working on this since earlier than yesterday. I'd like to hear your approach on the problem, maybe I have made some bad design decisions? – Nobody Aug 30 '14 at 7:59
• I need to think about this more, but my initial reaction is that the advantage of DECLARE_AND_TIE((index, element), indexElementPair); over auto& index = indexElementPair.first; auto& element = indexElementPair.second; is outweighed by its complexity. It seems like something I would be trying to convince myself I could accept after spending hours on it, rather than something I should actually use. – Michael Urman Aug 30 '14 at 13:34

### Reviewing the Implementation

Setting aside the question of whether this is the right approach, I really wanted to try to review the code you did write. I was worried that it required more of a learning curve than I could allocate time for, as I'm not yet familiar with using BOOST_PP_*. I can say now that your macro implementation is fairly straightforward to understand (assuming the BOOST_PP_* macros do what they have to in order for your code to work). I agree 100% with your focus on trying to provide clear compilation errors when the usage is incorrect. This is doubly important in cases like this with magical syntax. My biggest complaint is small: I cannot figure out how you chose the order of your different macros; they seem neither top-down, bottom-up, or alphabetically sorted. (Well, biggest complaint aside from the macros being macros.)

After all the side thoughts about handling an enumerate helper that seemed to return a std::pair, I was surprised to see nothing that appears to handle std::pair after all. Instead it would handle a 2-tuple, but this suggests it would be similarly unable to handle tie-ing inside a range-for over a std::map. It wouldn't surprise me if there's a way to extend the macros to handle this, but I do strongly suspect diminishing returns.

One of your questions was that of turning tests that fail to compile into actual live tests. Sometimes you can use techniques based on SFINAE in order to create such tests, but I'm not sure how to do that here. Everything I can think of would require turning your error cases into run-time exception throwing cases instead, which is a poor trade-off. And that still only handles things like mismatched sizes; it wouldn't handle a case you don't have a test for: reusing local names in multiple DECLARE_AND_TIE calls. All I can suggest here is considering using an #if defined(VERIFY_BAD_SYNTAX) instead of locally commenting the tests out to make it easier to verify them if you update your macros.

Anyway, definitely +1 for a cool question and clean code.

### Are There Better Ways?

Now I want to bring back the question of whether this is the right approach. I'm still thinking it is not. In the enumerate that returns a std::pair<Counter, Value>, I think we all agree it's way too much hidden complexity, and even probably too much usage complexity for its benefit. For the stronger motivating case where you want to iterate several collections in lock-step, I'm still not sold on the position of the code that the user has to write. Don't get me wrong; I do agree that

for(auto tuple: zip(numbers, names, ages, addresses))  {
: : :
}


is more comfortable to write than the corresponding manual reference declarations. But I'm not sure that it's more comfortable to read. Macros break the standard rules of the language. And while this is carefully crafted to be only a tiny burden to the reader, it is still yet another burden. It's a shame there's no way to use std::tie to name references to a std::tuple.

In a comment I proposed a magic unimplemented syntax that would instead read something like this:

for (auto rec : NamedTupleZip(numbers, names, ages, addresses)) {
// refer to rec.number, rec.name, etc.
}


Aside from the impossibility of this syntax, I think it places the complexity in the right place. Somehow NamedTupleZip would have to create a namedtuple-inspired struct that replaced the use of a std::tuple, and iterated over the respective collections like zip to populate each iteration's values.

You counter with two worries: one is that this could result in re-implementing the named tuple for each instance of a corresponding iteration, and that sometimes you are given a std::tuple rather than the opportunity to build your own. The second doesn't seem important to me; if the helper struct can construct from a std::tuple, it's easy to convert. The first may be a legitimate concern, but until we have an implementation, it can't be measured. And your point about ownership make sense; I would expect the original source (whether a zip'd collection or a referenced tuple) to maintain ownership of the actual data; perhaps this should be a NamedRefTuple instead.

To end with, I want to raise the question of whether you're using the correct data structures in the first place. In Python it's easy to create tons of arbitrary tuples by zipping structures in arbitrary order. But it's a lot less clear what the motivation is for doing that in C++. If you have a lot of cases where you're handling numbers, names, ages, addresses, it seems like you should already have a PersonInfo struct (or PersonInfoRef) that can construct from a std::tuple, and then all you need is zip:

for (PersonInfo rec : zip(numbers, names, ages, addresses))  {
// rec.number, rec.name, ...
}


If instead you want to handle lots of different arbitrary subsets, you have to weigh the cognitive load of approaches like DECLARE_AND_TIE vs. the traditional for loop.

While this is truly painful to read (and easy to use the wrong tuple index)

for(auto tuple: zip(numbers, names, ages, addresses))  {
auto &number = std::get<0>(tuple);
auto &name = std::get<1>(tuple);
auto &age = std::get<2>(tuple);
}


the pre-C++11 variant isn't so bad

const int i_max = std::min({numbers.size(), names.size(), ages.size(), addresses.size()});
for(int i = 0; i < i_max; ++i)  {
auto &number = numbers[i];
auto &name = names[i];
auto &age = ages[i];
}


and it has the advantage of calling out the termination case for mismatched collection lengths.

• I must admit that my ordering is just grown. The basic principles were: "public" interface first then symbols before they are used and in the order in which they are used. The problem about the "live" tests is more like: I would want to check if the compiler errors are sane but this is far too vague (would need a "compiler error testing framework" + some AI to evaluate the errors). I did not tests the duplicate naming problem because I found any errors related to that quite obvious (the compiler telling you that the name is already declared pointing to the macro invocation). – Nobody Aug 31 '14 at 20:34
• Regarding your second section: You are speeking of too much hidden complexity as a problem but the only problem I see there is in maintanence (/wider compiler support). The complexity is hidden to the user and you can hardly argue that it gives a measurable slowdown. As the name suggests I wanted a replacement for std::tie that does not involve copying/moving (though the name tie does not fit any more I think). From my two worries I would give the second more weight than the first. I can clearly imagine an implementation (that would again require a macro) but for NamedTupleZip. – Nobody Aug 31 '14 at 20:46
• Your last part focuses on the zip algorithm, which again is somewhat orthogonal to what I wanted to achieve. There are more instances than only enumerate and zip that could use DECLARE_AND_TIE. Basically everytime we are using tuples we can use better names than the indices to refer to the elements which would justify the aliasing and if we use more then two elements the DECLARE_AND_TIE would look cleaner (IMHO). Finally, your pre-C++11 version unnecessarily constrains the solution random access containers (and even there is non-optimal because of repeated indexing). – Nobody Aug 31 '14 at 20:52
• Despite the many concerns I commented now I have to +1 your answer. You have dealt with many of the aspects I wanted reviewed. I would really like to get into a more detailed discussion about this but I feel the comments are the wrong place for this. Would you be interested in such a discussion and if so should we take it to chat? – Nobody Aug 31 '14 at 20:54
• Unfortunately I don't typically have time available for chat, so here's more food for thought (I don't need the answers, but you might). I feel that you're balancing conflicting needs here. In the zip case, why is your related data in multiple containers, instead of records. For non-zip, why not work with the source of the tuple? DECLARE_AND_TIE solves only half the problem; however discussing ideas for solving the whole problem veers a bit from codereview's focus. (Your point about my pre-C++11 requiring random-access containers is true, but I lack the context to evaluate its importance.) – Michael Urman Sep 1 '14 at 13:11

## Usage remarks

If you're a single developer using this in your own project then I do not see any problems with the usage. After all it's a convenience for you. I can not comment on the design on the macros as I find them harder to read than what is worth spending my free time on (this is not specific to your macros, but to complex macros in general).

However if you are working in an environment where there are more developers than yourself, then I believe that this type of convenience macro is rather an inconvenience. In general I find that the further one goes to try to make C++ look like something it isn't by "inventing/implementing new syntax" using various macros, the more difficult it becomes to maintain over time. Developers come and go and the closer you stick to standard approaches to solving problems the better you will be able to handle new programmers on your project. As Michael Urman said in a comment on OP:

It seems like something I would be trying to convince myself I could accept after spending hours on it, rather than something I should actually use.

As you said yourself, this is syntactic sugar. But your sugar may be my vinegar as I'm not a python developer.

In any case, these are largely my personal opinions. Take with a grain of salt.

## Without macros

First of, I'm aware of the fact that this is not a typical review response so feel free to mod me into the ground. However based on a comment by op, I felt compelled to give this a shot without macros.

I would have liked to not use macros for this but I did not see any way around them. I tried to make the macro as user friendly as possible so if you don't want to wade through the implementation I'd still like to hear usage remarks. - Nobody

So here goes:

#include <iostream>
#include <vector>

template<typename T>
class enumerated{
private:
template<typename U>
struct bound_index{
bound_index(int idx, U& val) : index(idx), value(val) {}

int index;
U& value;
};

typedef decltype(std::begin(*((T*)(0)))) it_t;

struct iterator{
iterator(it_t idx) : m_it(idx){}

it_t m_it;
int m_idx{0};

bound_index<decltype(*m_it)> operator *(){
return bound_index<decltype(*m_it)>(m_idx, *m_it);
}

bool operator == (const iterator& rhs){ return m_it == rhs.m_it; }
bool operator != (const iterator& rhs){ return !(*this == rhs); }

iterator operator ++(){
iterator ret = *this;
m_it++;
m_idx++;
return ret;
}

iterator operator ++(int){
m_it++;
m_idx++;
return *this;
}
};

public:
enumerated(T& t) : m_data(t) {}

iterator begin(){ return iterator(std::begin(m_data)); }
iterator end(){ return iterator(std::end(m_data)); }

private:
T& m_data;
};

// This allows template argument deduction to work so that you don't have to
// specify enumerate<std::vector<double>> etc
template<typename T>
enumerated<T> enumerate(T& t){ return enumerated<T>(t); }

int main(){
std::vector<double> v{0.1, 2.1, 43.1, 4.4};
for(auto entry : enumerate(v)){
std::cout<<"Index = "<<entry.index<<" Value = "<< entry.value<<std::endl;
}
}


I'm aware that this is not perfect and will probably fail for types other than std::vector but it does illustrate a technique. It can probably be done in a prettier way too. It doesn't provide exactly the same convenience as OP but it is close IMHO and it doesn't use macros.

If one wishes to get the same simplicity as OP one may use a macro in the tune to:

#define DECLARE_AND_TIE(IDX, VAL, ENTRY) auto& IDX = ENTRY.index; auto& VAL = ENTRY.value;


Oops I realize after writing all this that the above example code misses some functionality that OP's macros have. But it solves the motivational example that was given to get the index and value during the iteration of a range based for loop.

• Your alternative implementation solves enumerate and not the part that I wanted to solve which would integrate with tuples in general. By usage remarks I meant things like: "I don't understand how to use your code correctly and the error message my compiler gives is no help." – Nobody Aug 31 '14 at 10:58
• To be honest, your code looks easy to use so there is no problem on that aspect. However my sentiments are the same as those of the comments on OP. I also believe that your motivational example is misleading for the actual question. How would your macro solve the motivational example? – Emily L. Aug 31 '14 at 11:09
• Admittedly it is a bit misleading. I assumed the existance of an implementation of enumerate that returns a tuple. Take a look at "More motivation" which gives another example (also inspired by Python) which assumes an existing zip function that returns tuples. – Nobody Aug 31 '14 at 11:28
• Okay I can think of a way using lambda functions to do this without macros. Are you interested in seeing it? – Emily L. Aug 31 '14 at 12:15
• It sounds more restrictive but I am keen on non macro implementations. – Nobody Aug 31 '14 at 12:20