I wrote a test that parses links from a web page, and in a loop, clicks on each of them, and then returns to the main page. But, each iteration is accompanied by parsing the entire web page again and again and overwriting the links array, although only one link is needed per iteration. I understand that this is inefficient. How can I optimize this?

I tried parsing the links only once and then iterating through them in a loop. After the first iteration, it goes back (to the main page) and tries to click on the second link, but it's not interactive (I think it's because of the web elements that are stored in the links array changing every time you go to the page).

def setUp(self):
    self.driver = webdriver.Chrome()

def test_01(self): 
    driver = self.driver
    links = []
    links = driver.find_elements_by_css_selector("a")
    for i in range(len(links)):
        links = driver.find_elements_by_css_selector("a")
  • 1
    \$\begingroup\$ I'm not convinced that what you want to do even makes sense. When you click on a link, then go back, there is no guarantee that the page has the same content as it originally did. \$\endgroup\$ – 200_success Sep 15 '19 at 3:07
  1. When you navigate away from the page the WebElements get invalidated as DOM changes therefore I would recommend going for List Comprehension and convert the list of WebElements into the list of links text.
  2. Once done you should be able to use normalize-text() function just in case the text belongs to child element or has starting/trailing spaces
  3. And last but not the least consider using Explicit Wait just in case the link is populated via AJAX call.

Example code:

links = [link.text for link in driver.find_elements_by_css_selector("a")]
for link in links:
    WebDriverWait(driver, 10) \
        (By.XPATH, "//a[normalize-space()='" + link + "']"))).click()
    # do what you need here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.