Assigning an array of different (but convertible) type to concurrency::array in C++AMP

Question

I am creating a vector class which can be manipulated on the GPU and I am using C++AMP for the GPU accelerated code.

I am wondering the most efficient way of assigning elements of a different type to a concurrency::array in C++AMP. There are two ways that I can think of (but there may be more):

Firstly, one could use a simple sequential for loop and use the concurrency::array_view object to assign the data members and then call concurrency::array_view::synchronize at the end to ensure the data is stored on the accelerator for subsequent operations, this would look like following:

template <typename T>
CVector& operator=( T tArray[Size] )
{
    static_assert( std::is_convertible<T, NumType>::value, "Cannot assign a vector from an array of non-convertible type" );

    concurrency::array_view<NumType, 1> avThis( m_numArray.view_as( m_numArray.extent ) );

    for( std::size_t s = 0; s < Size; ++s )
    {
         avThis(s) = static_cast<NumType>(tArray[s]);
    }

    avThis.synchronize();
    return *this;
}

Alternatively, one could construct a concurrency::array_view from the array and then use a concurrency::parallel_for_each to assign the data members as follows:

template <typename T>
CVector& operator=( T tArray[Size] )
{
    static_assert( std::is_convertible<T, NumType>::value, "Cannot assign a vector from an array of non-convertible type" );

    concurrency::array_view<const T, 1> avArray( Size, tArray );

    concurrency::parallel_for_each( m_numArray.extent, [=, &m_numArray]( concurrency::index<1> index ) restrict(amp) {
         m_numArray[index] = static_cast<NumType>(avArray[index]);
    } );

    return *this;
}

Which of these methods results in better performance or are they equal?

---

I profiled the code, and using the Microsoft Concurrency Visualizer, produced the following graph:

Answer 1

As with almost any performance related question, the correct answer is probably to do some profiling to get solid answers.

That said, it strikes me as unlikely that attempting to do the copy in parallel will usually make much difference. In a typical case, copying to the accelerator takes place over a PCIe bus. In that typical case, even a single CPU core will have substantially higher memory bandwidth than that bus, so the copy will be a "hurry up and wait" situation; using more than one CPU will basically means hurrying even faster so you can wait even longer.

In the case of something like an AMD APU, the parallel copy stands at least a little more chance of improving performance. Even here I'd have substantial doubts though. In particular, a single core can still normally (more than) saturate the processor's bandwidth to main memory, so doing the copy in parallel still won't gain anything.

That leads us to a much more general conclusion: if your processing is memory bound to start with, parallel execution is unlikely to improve performance unless you do it on hardware that has multiple sockets and each socket has a separate memory controller. In this case, the code will probably still be limited by bandwidth to memory, but you've doubled the bandwidth to memory which can roughly double the speed of the code.