Overarching Problem
I am creating a threadsafe queue that will act as a shared buffer between two threads for a game I am developing. I need it to be threadsafe so that one thread can throw messages into the queue, and second thread can take them out for processing in real time. I have decided in order to really learn about concurrency and threading I would like to create my own threading library with the required synchronization structures(starting off with a mutex that is implemented as a spinlock). For now I am using the Poco Threading library in order to test and develop my mutex. I will move forward to creating a Windows/Linux threading environment after I have my basic synchronization structures.
Proposed Solution (mutex/spinlock)
Note: Sender and Receiver do not actually do any sending or receiving, or any queue work, they are just simply operating on numbers in tight for-loops to simulate work. I did not want to add extra variables to the test before I verified that my mutex implementation was working as expected.
Mutex.hpp
#ifdef WIN32
#include <Windows.h>
#define TARG_PLATFORM_WINDOWS
#else
#include <unistd.h>
#define TARG_PLATFORM_LINUX
#endif
//Constant values
const int MUTEX_LOCKED = 1;
const int MUTEX_UNLOCKED = 0;
class Mutex
{
private:
volatile unsigned long long interlock;
public:
Mutex();
~Mutex();
void lock();
void unlock();
};
Mutex.cpp
#include "Mutex.hpp"
Mutex::Mutex()
{
interlock = 0;
}
Mutex::~Mutex()
{
}
void Mutex::lock()
{
#ifdef TARG_PLATFORM_WINDOWS
while(this->interlock == 1 || InterlockedCompareExchange(&this->interlock, 1, 0) == 1);
#endif
#ifdef TARG_PLATFORM_LINUX
while(this->interlock == 1 || __sync_lock_test_and_set(&this->interlock, 1) == 1);
#endif
}
void Mutex::unlock()
{
this->interlock = 0;
#ifdef TARG_PLATFORM_WINDOWS
#endif
#ifdef TARG_PLATFORM_LINUX
#endif
}
Testing code
Note: For easy reading I simply included one threads test code. The other thread has the exact same code except it is the "Receiver" and it only simply locks and unlocks no sleep() is called in thread 2. This should make it so they are both in similar starting areas when the work begins. Basically I am trying to make each thread do some work and then unlock the mutex to allow the other thread to do some work.
virtual void run()
{
int table[2000000];
std::cout<<"Thread1 Started"<<std::endl;
mutex->lock();
sleep(3) //Allow threads to both catch up to eacother
mutex->unlock();
mutex->lock();
for(int i = 0; i < 2000000; i++)
{
table[i] = i/2;
}
std::cout<<"Sender block 1 done"<<std::endl;
mutex->unlock();
mutex->lock();
for(int i = 0; i < 2000000; i++)
{
table[i] = i/2;
}
std::cout<<"Senderblock 2 done"<<std::endl;
mutex->unlock();
mutex->lock();
for(int i = 0; i < 2000000; i++)
{
table[i] = i/2;
}
std::cout<<"Sender block 3 done"<<std::endl;
mutex->unlock();
//dequeue a msg
}
My Concerns
When I run the test, the threads seem to execute in order and not taking turns:
Thread1 started Thread2 started Sender Block 1 done Sender block 3 done Sender block 2 done Receiver block 2 done Receiver Block 1 done Receiver block 3 done
I thought thread 2 would be able to acquire the lock before thread 1 could unlock() and lock() the mutex again. For the sake of testing I added a sleep for 100 milliseconds in the unlock() function in order to give the other thread time to acquire the lock. This worked and they alternated with the following output:
Thread1 started Thread2 started Sender Block 1 done Receiver Block 1 done Sender block 2 done Receiver block 2 done Sender block 3 done Receiver block 3 done
It seems that the re-locking is too fast for the threads to take turns and I am concerned this may starve thread2 in a real application. But I do not want a sleep() in the unlock() function as this is a huge performance hit and causes context switching.
I did some reading and some people did performance tests of locking and relocking their mutex object so I tried the same for mine. It took 8-10 milliseconds for it to lock and unlcok 1 million times in a tight for loop. This seems to fast compared to other things I have read. Perhaps I am missing something?
Ultimate Questions
Should I be using a different type of lock? Is something off with my Mutex implementation? Is something invalidating the test I have written? If so what kind of test would you recommend?
Note I know this seems kind of like a Stack Overflow question but I figured since I had a solution and I wanted more of a review and possible suggestions on a solution that involves modifying code already written, this would be a better place for the question.
