Finding solutions on GitHub and Stack Overflow

Question

The first part of the code opens Chrome and navigates to GitHub. If I don't find my solution, I open a new tab with ctrlt. Then my code goes to Stack Overflow and searches there for solutions.

from selenium import webdriver
from selenium.webdriver.common.keys import *
import time
import keyboard

q = input("what is your programming question : ")+(' python')
class github :
    path = "c:/users/admin/appdata/local/programs/python/python38-32/chromedriver.exe"
    driver = webdriver.Chrome(path)

    driver.get("https://github.com/")
    question = driver.find_element_by_name("q")
    time.sleep(3)


    question.send_keys(q)
    question.send_keys(Keys.RETURN)

    while True:
        if keyboard.is_pressed("ctrl+t"):
            driver.switch_to_window(driver.window_handles[-1])
            driver.get("https://stackoverflow.com/")
            question_2 = driver.find_element_by_name("q")

            question_2.send_keys(q)
            question_2.send_keys(Keys.RETURN)
            break

Answer 1

Selenium

Selenium imposes a lot of overhead and complexity that you don't need to deal with. If you needed to scrape, use requests + beautifulsoup; but you don't: StackExchange has a good API, so you should strongly prefer using that. It will get you results more efficiently and the results will be more accurate and less fragile.

PEP8

class github :

should be

class Github:

Answer 2

For simply content scraping without JavaScript and ajax content try scrapy for best practices. Scrapy uses Python classes by default as it is a Python framework.

Easy tutorial to learn Scrapy:

Scrapy Tutorial on Youtube

Official site of ScrappingHub

Selenium is good for scraping dynamic content and causes unnecessary overhead as mentioned in above answer.

For above code: Try avoiding time.sleep and use EC.presence_of_element_located and similar functions to obtain desired behavior. Selenium Waits