PPL and AMP performing worse than sequential transform

Question

I wrote the following short test code to test the performance of C++AMP and the PPL libraries against the sequential STL implementation of std::transform. To my surprise, both C++AMP and PPL implementations were significantly worse than the sequential implementation (C++AMP: 128ms, PPL: 51ms, Sequential: 25ms). This pattern was the case with int, float and double data types.

I expected for small sizes (perhaps less than a few thousand) that the sequential code would be fastest as there is a significant time delay in copying data from the CPU to the GPU RAM and there is a slight delay in thread-starting etc. for the PPL, however I didn't expect for large sizes (100000+) that the sequential code would win.

I used the following code to measure performance and compiled with full optimisations in Visual Studio 2013:

#include <amp.h>
#include <iostream>
#include <numeric>
#include <random>
#include <assert.h>
#include <functional>
#include <chrono>
const std::size_t size = 30737418;

using namespace concurrency;

//----------------------------------------------------------------------------
// Program entry point.
//----------------------------------------------------------------------------
int main( )
{
    accelerator default_device;
    std::wcout << "Using device : " << default_device.get_description( ) << std::endl;
    if( default_device == accelerator( accelerator::direct3d_ref ) )
        std::cout << "WARNING!! Running on very slow emulator! Only use this accelerator for debugging." << std::endl;

    std::mt19937 engine;
    std::uniform_int_distribution<int> dist( 0, 10000 );

    std::vector<int> vecTest( size );
    std::vector<int> vecTest2( size );
    std::vector<int> vecResult( size );

    for( int i = 0; i < size; ++i )
    {
        vecTest[i] = dist( engine );
        vecTest2[i] = dist( engine );
    }

    std::vector<int> vecCorrectResult( size );

    std::chrono::high_resolution_clock clock;
    auto beginTime = clock.now();

    std::transform( std::begin( vecTest ), std::end( vecTest ), std::begin( vecTest2 ), std::begin( vecCorrectResult ), std::plus<int>() );

    auto endTime = clock.now();
    auto timeTaken = endTime - beginTime;

    std::cout << "The time taken for the sequential function to execute was: " << std::chrono::duration_cast<std::chrono::milliseconds>(timeTaken).count() << "ms" << std::endl;

    beginTime = clock.now();

    concurrency::array_view<const int, 1> av1( vecTest );
    concurrency::array_view<const int, 1> av2( vecTest2 );
    concurrency::array_view<int, 1> avResult( vecResult );
    avResult.discard_data();

    concurrency::parallel_for_each( avResult.extent, [=]( concurrency::index<1> index ) restrict(amp) {
        avResult[index] = av1[index] + av2[index];
    } );

    avResult.synchronize();
    endTime = clock.now();
    timeTaken = endTime - beginTime;

    std::cout << "The time taken for the AMP function to execute was: " << std::chrono::duration_cast<std::chrono::milliseconds>(timeTaken).count() << "ms" << std::endl;
    std::cout << std::boolalpha << "The AMP function generated the correct answer: " << (vecResult == vecCorrectResult) << std::endl;

    beginTime = clock.now();

    concurrency::parallel_transform( std::begin( vecTest ), std::end( vecTest ), std::begin( vecTest2 ), std::begin( vecResult ), std::plus<int>() );

    endTime = clock.now();
    timeTaken = endTime - beginTime;

    std::cout << "The time taken for the PPL function to execute was: " << std::chrono::duration_cast<std::chrono::milliseconds>(timeTaken).count() << "ms" << std::endl;
    std::cout << "The PPL function generated the correct answer: " << (vecResult == vecCorrectResult) << std::endl;

    return 0;
}

Is there anything I could do to improve the parallel performance or is it simply that adding is such a quick operation that the overhead of parallelization will always be greater than the speedup?

Answer 1

I think in this case parallel execution (at least off the CPU) is unlikely to give a speedup. The problem is fairly simple: the operation you're carrying out (addition) is so simple that the bandwidth to memory is the controlling factor in the overall speed.

With the operation happening serially on the CPU, you get an answer as fast as you can read in the data from memory.

For the AMP version, the data is read from memory, then written to the GPU's memory, then the GPU reads the data back to produce a result, and finally the data gets written back to where the CPU can see it.

To see where AMP can provide an advantage, you're almost certainly going to have to do more operations on the data, so the overall time is substantially greater than the time to just fetch the raw data from memory.