I am working on a project right now, where we need to search different APIs, then clean & persists data to database and finally pushing to Elastic searchElasticsearch.
The high level code looks like belowthis:
class BaseParser(object):
def __init__(self, source):
self.url = source.url
self.api_key = source.api_key
def _prepare_auth_header(self):
headers = {
"Authorization": "Basic {}".format(self.api_key)
}
return headers
def get_response(self):
raise NotImplementedError()
class UnitedexchangeParser(BaseParser):
def get_response(self):
headers = self._prepare_auth_header()
response = requests.get(self.url, headers=headers)
return response.json()
class DubaiParser(BaseParser):
def get_response(self):
headers = self._prepare_auth_header()
response = requests.get(self.url, headers=headers)
return response.json()["data"]
class ParserFactory(object):
def parse(self, source):
if source.name == "Unitedexchange":
parser = Unitedexchange(source)
elif source.name == "Dubai":
parser = DubaiParser(source)
return parser.get_response()
The application is working fine, but we are adding more and more new sources which leads to change ParserFactoryParserFactory
with same block of code.
How I can I improve the lay outlayout and design such a way so we do minimal change for adding new parser? And also performancePerformance is also an issue, and right now we are loading lots of individual classes which basically doing the same thing, it. It would be great if you can give some hint on that, too.
Thanks in advance.