4
\$\begingroup\$
struct __fifo_node_t {
        void *next;
};
typedef struct __fifo_node_t fifo_node_t;

struct __fifo_list_t {
        void *next;
        void *tail;
};
typedef struct __fifo_list_t fifo_list_t;

static inline void __fifo_list_push(fifo_list_t *head, fifo_node_t *node)
{
        fifo_node_t *prev;

        node->next = head;
        prev = atomic_xchg64(&head->tail, node);
        prev->next = node;
}

static inline fifo_node_t *__fifo_list_pop(fifo_list_t *head)
{
        fifo_node_t *node;
        while ((node = ACCESS_ONCE(head->next)) != head) {
                fifo_node_t *next = ACCESS_ONCE(node->next);
                if (cmpxchg_eq(&head->next, node, next) == node) {
                        if (cmpxchg_eq(&head->tail, node, head) != node &&
                            next == head)
                                while (ACCESS_ONCE(head->next) == next)
                                        head->next = ACCESS_ONCE(node->next);
                        return node;
                }
        }
        return NULL;
}

static inline void __fifo_list_init(fifo_list_t *head)
{
        head->next = head->tail = head;
}

atomic_xchg64 - Atomic exchange 64bit with new value and return the old value

cmpxchg_eq - Atomic compare and exchange 64bit with new value and return the old value

ACCESS_ONCE - (*(volatile typeof(x) *)&(x))

Can you review it? Is this implementation thread-safe ?

Thank you.

\$\endgroup\$

1 Answer 1

3
\$\begingroup\$

The meaning of lock-free

Although your program is free of locks in the traditional sense, "lock free programming" typically describes a style of programming with certain guarantees. One of these guarantees is that even if one or more threads are suspended indefinitely, other threads can continue to make progress. This wikipedia article on Non-blocking algorithms describes "lock-free" and "wait-free" programming in more detail.

In your code, if one thread is permanently suspended between these two lines:

    prev = atomic_xchg64(&head->tail, node);
    prev->next = node;

then some other thread will spin forever in this loop:

                            while (ACCESS_ONCE(head->next) == next)
                                    head->next = ACCESS_ONCE(node->next);

and furthermore, no nodes past the node that the suspended thread "half-pushed" will be able to be popped.

ABA problem

Additionally, your code falls victim to the ABA problem. Consider an initial situation with two nodes in your FIFO:

A -> B    (head->next = A, head->tail = B)

Suppose thread #1 calls __fifo_list_pop(), and gets to this line:

            if (cmpxchg_eq(&head->next, node, next) == node) {

Here, node will be A, and next will be B, and the code is about to atomically change head->next from A to B.

Now suppose thread #1 is descheduled, and in the meantime, other threads pop both A and B off the FIFO, and then push A and C back on the FIFO. Now, your FIFO looks like this:

A -> C    (head->next = A, head->tail = C)

If thread #1 now wakes up, it will perform the atomic exchange, causing head->next to point at B which isn't even alive any more:

A -> B    (but B was already popped and is now dead)
\$\endgroup\$
4
  • \$\begingroup\$ Yes, you're right. \$\endgroup\$
    – ShadowStar
    Mar 27, 2017 at 4:14
  • \$\begingroup\$ I always working on kernel mode (no-preemption, disabled-irq) or cavium octeon simple executive mode, and ignored scheduling of process. \$\endgroup\$
    – ShadowStar
    Mar 27, 2017 at 4:20
  • \$\begingroup\$ @ShadowStar If you have no preemption and interrupts disabled, then the first section about deadlocking won't happen. But you can still get the ABA problem if you encounter a rare set of timings. Basically, there is a memory delay between a write on one core and its visibility on another core, and if the timings work out just the wrong way, the ABA problem will occur. \$\endgroup\$
    – JS1
    Mar 27, 2017 at 5:36
  • \$\begingroup\$ Yes, there is ABA problem in theory. \$\endgroup\$
    – ShadowStar
    Mar 27, 2017 at 5:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.