2
\$\begingroup\$

I am creating a vector class which can be manipulated on the GPU and I am using C++AMP for the GPU accelerated code.

I am wondering the most efficient way of assigning elements of a different type to a concurrency::array in C++AMP. There are two ways that I can think of (but there may be more):

Firstly, one could use a simple sequential for loop and use the concurrency::array_view object to assign the data members and then call concurrency::array_view::synchronize at the end to ensure the data is stored on the accelerator for subsequent operations, this would look like following:

template <typename T>
CVector& operator=( T tArray[Size] )
{
    static_assert( std::is_convertible<T, NumType>::value, "Cannot assign a vector from an array of non-convertible type" );

    concurrency::array_view<NumType, 1> avThis( m_numArray.view_as( m_numArray.extent ) );

    for( std::size_t s = 0; s < Size; ++s )
    {
         avThis(s) = static_cast<NumType>(tArray[s]);
    }

    avThis.synchronize();
    return *this;
}

Alternatively, one could construct a concurrency::array_view from the array and then use a concurrency::parallel_for_each to assign the data members as follows:

template <typename T>
CVector& operator=( T tArray[Size] )
{
    static_assert( std::is_convertible<T, NumType>::value, "Cannot assign a vector from an array of non-convertible type" );

    concurrency::array_view<const T, 1> avArray( Size, tArray );

    concurrency::parallel_for_each( m_numArray.extent, [=, &m_numArray]( concurrency::index<1> index ) restrict(amp) {
         m_numArray[index] = static_cast<NumType>(avArray[index]);
    } );

    return *this;
}

Which of these methods results in better performance or are they equal?


I profiled the code, and using the Microsoft Concurrency Visualizer, produced the following graph:

Concurrency visualization

\$\endgroup\$

1 Answer 1

2
\$\begingroup\$

As with almost any performance related question, the correct answer is probably to do some profiling to get solid answers.

That said, it strikes me as unlikely that attempting to do the copy in parallel will usually make much difference. In a typical case, copying to the accelerator takes place over a PCIe bus. In that typical case, even a single CPU core will have substantially higher memory bandwidth than that bus, so the copy will be a "hurry up and wait" situation; using more than one CPU will basically means hurrying even faster so you can wait even longer.

In the case of something like an AMD APU, the parallel copy stands at least a little more chance of improving performance. Even here I'd have substantial doubts though. In particular, a single core can still normally (more than) saturate the processor's bandwidth to main memory, so doing the copy in parallel still won't gain anything.

That leads us to a much more general conclusion: if your processing is memory bound to start with, parallel execution is unlikely to improve performance unless you do it on hardware that has multiple sockets and each socket has a separate memory controller. In this case, the code will probably still be limited by bandwidth to memory, but you've doubled the bandwidth to memory which can roughly double the speed of the code.

\$\endgroup\$
2
  • \$\begingroup\$ I took your advice and profiled it (albeit rather basically); and I found something interesting, for small sizes (I tried with 10000) the sequential for-loop consistently outperformed the parallel for-each loop by a factor of almost x10. When I did it with medium sized sizes the performance was identical, but when I did it with high end sizes (I tried it with 83741823) the parallel_for_each performed almost 8x faster than the sequential for loop. \$\endgroup\$ Jul 9, 2014 at 14:54
  • \$\begingroup\$ I've added a concurrency visualization from Visual Studio to my question in case you find it interesting :) \$\endgroup\$ Jul 9, 2014 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.