I am parsing a set of lines from a file, after reading the file content with Parser.read_file()
I apply a set of methods to solve different issues each line may contain, like removing new lines, comments etc, and set a list of lines in the parser self.lines
instance variable.
Filter methods are defined in Parser.filter
as a list of names and executed after the file is read with self._sanitize()
.
import re
import os
import io
from operator import methodcaller
class Parser:
# List of filters
filters = ['_filter_comments',
'_filter_new_lines',
'_filter_empty_entries']
def __init__(self, **kwargs):
self.lines = kwargs.get('lines', [])
self.file_path = kwargs.get('file_path', "data/data_ts.utf8")
self.lines_count = kwargs.get('entries_count', None)
def read_file(self):
""" Reads file data applying filters from self.filters """
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
with io.open(os.path.join(__location__, self.file_path), "r",
encoding='utf-8') as f:
self.lines = f.readlines()
self._sanitize()
def _sanitize(self):
for fn in self.filters:
f = methodcaller(fn)
f(self)
def _filter_comments(self):
""" remove lines starting with # or #! """
self.lines = [line for line in self.lines
if not line.startswith(("#", "#!"))]
def _filter_new_lines(self):
self.lines = [line.strip('\n') for line in self.lines]
def _filter_empty_entries(self):
self.lines = [line for line in self.lines if line.strip()]
Issues
- I think that the pythonic way to write those filters in one line as
[line.strip('\n') for line in self.lines if line.strip() and if not line.startswith(("#", "#!")]
would make it harder to read and unit test them or it would be better that way? Should the
_sanitize
method be called after reading the file or would it make more sense to put it in the setter property ofself.lines
like:@lines.setter def lines(self, lines): self.lines = lines self._sanitize()
Is there any better option to arrange the above code or a design pattern like putting the sanitize method outside the Parser class and receiving a list of lines and returning the sanitized list instead of working always with
self.list
?