I am working on a project right now, where we need to search different APIs, then clean & persists data to database and finally pushing to Elastic search.
The high level code looks like below:
class BaseParser(object):
def __init__(self, source):
self.url = source.url
self.api_key = source.api_key
def _prepare_auth_header(self):
headers = {
"Authorization": "Basic {}".format(self.api_key)
}
return headers
def get_response(self):
raise NotImplementedError()
class UnitedexchangeParser(BaseParser):
def get_response(self):
headers = self._prepare_auth_header()
response = requests.get(self.url, headers=headers)
return response.json()
class DubaiParser(BaseParser):
def get_response(self):
headers = self._prepare_auth_header()
response = requests.get(self.url, headers=headers)
return response.json()["data"]
class ParserFactory(object):
def parse(self, source):
if source.name == "Unitedexchange":
parser = Unitedexchange(source)
elif source.name == "Dubai":
parser = DubaiParser(source)
return parser.get_response()
The application is working fine, but we are adding more and more new sources which leads to change ParserFactory with same block of code.
How I can improve the lay out and design such a way so we do minimal change for adding new parser? And also performance is also an issue, right now we are loading lots of individual classes which basically doing the same thing, it would be great if you can give some hint on that too.
Thanks in advance.