5
\$\begingroup\$

I would like to write a small helper to filter datasets. Each dataset is a dictionary and some filters I want to apply are common to all the datasets.

That my needs, I wrote this following pattern:

class Dataset(dict):
    """Just a regular dictionary that can be filtered using filters."""
    _filters = []
    def add_filter(self, filter):
        self._filters.append(filter)

    def apply_filters(self):
        for f in self._filters:
            f(self)

class Filter(object):
    """A callable class that filter a dictionary with the _filter method""".
    def __init__(self, keys, **kwargs):
        self.keys = keys
        self.kwargs = kwargs

    def __call__(self, dictionary):
        """Filter a dictionary."""
        self._filter(dictionary)

    def _filter(self, dictionary):
        """Must be overloaded."""
        pass

class PassTrough(Filter):
    pass

class DropIfEmpty(Filter):
    def _filter(self, dictionary):
        for k in self.keys:
            if not dictionary.get(k):
                dictionary.pop(k)

class MoveInSubkey(Filter):
    def _filter(self, dictionary):
        d = dictionary[self.kwargs['subkey']] = dict()
        for k in self.keys:
            d[k] = dictionary[k]
            dictionary.pop(k)

ds = Dataset({'foo': 42, 'bar': 23, 'min': 0, 'max': 100, 'empty': None})

ds.add_filter(PassTrough([]))
ds.add_filter(DropIfEmpty(['empty']))
ds.add_filter(MoveInSubkey(['min', 'max'], subkey='range'))

ds.apply_filters()

Is that implementation ok, overkill, or should I turn myself to an existing data-filtering-engine that I don't know the existence yet?

Later on, I would like to use a more complex definition for my Dataset class where the filters I want to apply can be initially set like this:

class Foo(DataReader):
    __datasource__ = 'meta/foo.yml'
    __filters__ = [
        DropIfEmpty('foo', 'bar'),
        FilterThis('baz'),
        MoveInSubkey('range', ('min', 'max')),
    ]
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.