0
\$\begingroup\$

I created an application using multiple threads defined in multiple class files which read and write into a "shared variable" (a dictionary).

To ensure thread-safety, I am using a Lock, and I pass it as argument to my Thread class next to my variable.

What bothers me is that two objects which are intrinsically linked have to be passed to my class, while ideally only the shared variable should be. This would reduce the code and bring greater certainty while avoiding confusion in the parameters.

Here is how my program looks:

inc_thread.py
from threading import Thread


class IncThread(Thread):

    def __init__(self, shared_dict, lock):
        Thread.__init__(self, daemon=True)
        self._shared_dict = shared_dict
        self._lock = lock

    def run(self):
        while 1:
            with self._lock:
                self._shared_dict['some_value'] += 1
dec_thread.py
from threading import Thread


class DecThread(Thread):

    def __init__(self, shared_dict, lock):
        Thread.__init__(self, daemon=True)
        self._shared_dict = shared_dict
        self._lock = lock

    def run(self):
        while 1:
            with self._lock:
                self._shared_dict['some_value'] -= 1
main.py
import time

from threading import RLock
from dec_thread import DecThread
from inc_thread import IncThread


if __name__ == '__main__':
    lock = RLock()
    shared_dict = {'some_value': 0}

    t1 = DecThread(shared_dict, lock)
    t2 = IncThread(shared_dict, lock)

    t1.start()
    t2.start()

    # Ensure equity regarding wich thread started first
    with lock:
        shared_dict['some_value'] = 0

    while 1:
        print(shared_dict['some_value'])
        time.sleep(1)

Is it the correct way to do it?

What I also thought:

  • Subclass dict and override get / set methods with an internal lock, but it becomes complicated when I want to set other datastructures (nested dict, list, set, etc.) as values.

  • Using Queue but it does not seem to fit my needs, as it is by default blocking for put / get while my design is more of "fire and forget".

\$\endgroup\$
  • \$\begingroup\$ This code sound hypothetical to me, which makes it off-topic for Code Review. Is that your real code? \$\endgroup\$ – Mathias Ettinger Jul 13 '16 at 15:11
  • 1
    \$\begingroup\$ Also, as regard to your question, why don't you make an class LockedDict that contains the lock and whose __getitem__, __setitem__ and __delitem__ use before delegating to the dict equivalent methods? You can even define a mixin that you can use with whatever container you like, like class LockedDict(Locked, dict): pass. \$\endgroup\$ – Mathias Ettinger Jul 13 '16 at 15:12
2
\$\begingroup\$

Your concern seems legit. Fusing data that needs thread-safe operations and means of those operations would be nice.

However, this:

Subclass dict and override get / set methods with an internal lock

will not work, because += - which you use - will not be atomic.

Things get worse with nested data structures: we actually have several objects and some of them carry references to others. If there are also some more references to those objects - how do we check that? Example:

a = [1,2,3]
b = {'a' : a} # or some crafty wrapper on top of that

If we make sure that every b['a'] is atomic, a can still be changed directly.

Making a wrapper that accepts mutation function and executes it in a locked context would not really work if you need in-place operations, not replacements of the whole object.

What you can do is take a closer look at context managers and try to create an object that is both dictionary and context manager.

Or simply do something like:

class FlexibleDict(dict):
    pass

shared_dict = FlexibleDict(shared_dict)
shared_dict.lock = lock

and pass that around.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.