For simply search without javascript and ajax content try [scrapy][1] for best practices. Easy tutorial to learn scrapy [ScrapingHub Tutorial][2][Official site of ScrappingHub][3] Selenium is good for scraping dynamic content and causes unnecessary overhead as mentioned in above answer. For above code: Try avoiding `time.sleep` and use `EC.presence_of_element_located` and similar functions to attain desired behavior. [Selenium Waits][4] [1]: https://scrapy.org/ [2]: https://www.youtube.com/watch?v=vkA1cWN4DEc&list=PLZyvi_9gamL-EE3zQJbU5N3nzJcfNeFHU [3]: https://www.scrapinghub.com/learn-scrapy#get-started-scrapy-tutorials [4]: https://selenium-python.readthedocs.io/waits.html