5
votes
Finding solutions on GitHub and Stack Overflow
For simply content scraping without JavaScript and ajax content try scrapy for best practices. Scrapy uses Python classes by default as it is a Python framework.
Easy tutorial to learn Scrapy:
Scrapy ...
- 462
5
votes
Finding solutions on GitHub and Stack Overflow
Selenium
Selenium imposes a lot of overhead and complexity that you don't need to deal with. If you needed to scrape, use requests + ...
- 65.6k
4
votes
Extract flight arrival data from web
First: As @PavloSlavynskyy warns, FlightRadar24 has attempted to make it very clear that they do not want you to scrape their site or use their API. Among other verbiage, they say:
Copyright (c) 2014-...
- 65.6k
4
votes
Accepted
Find DivTag element in list and click when innerHTML matches account name
To eliminate the need for the flag you could do it like this:
...
- 156
3
votes
Accepted
Safely open a list of links in Firefox
Rather than implementing this yourself, Python by default has webbrowser. This allows you to open one link in a; new tab, or new window. And so, to open all links ...
- 43.5k
3
votes
Accepted
Recursively checking the css class of an iFrame Element
Code structure wise, the recursive pattern in JavaScript/Selenium and Protractor code is a pretty common one. The biggest problem here is that, you don't have a recursive cycle exit condition - if <...
- 17.3k
3
votes
Select fields from a form by ID or Xpath and fill its value
You use a lot of absolute xpaths. It's a bad practice, since those are costly to maintain and looks bad. Try to modify them to relative xpath's or preferably css selectors.
Instead of implicit wait, ...
- 191
2
votes
Accepted
Beating the queue
Web-scraping related improvements
I would replace the "by class" searches with more concise CSS selectors
.text.strip() could be replaced with ...
- 17.3k
2
votes
Project : My Anime Finder
It's looking good and it's an interesting project. I'm not familiar with selenium, but since you create a Class for each anime show sharing most of the code, you could implement a parent anime show ...
- 322
2
votes
Program to automate clicking on the new button on a chat website until the username matches certain conditions
Your third part is wrong, I think.
nonCompatible = 'female' not in name
will only run against the value of name at the time ...
- 2,306
2
votes
Reusable webdriver framework using SOLID design principles
I'm not really sure that there is any good argument having these static fields, especially when it comes to unit testing the WebDriver class.
...
- 1,023
2
votes
WaitFor method for website automation testing with Selenium WebDriver
This is a bit of a reimagining, but I think it will help.
Selenium actually has a smart wait helper package called DotNetSeleniumExtras.WaitHelpers.
With it, you ...
- 256
2
votes
Accepted
WaitFor method for website automation testing with Selenium WebDriver
WaitFor
This method name is kinda weird for me
If I would be your API's consumer then I would except from this naming that it would return with a ...
- 9,879
1
vote
Automating TypeRacer using selenium
I encourage you to examine the site in closer detail. So many instances of Selenium use are at the wrong layer of abstraction and this is no different.
The site communicates with XHR request; one ...
- 65.6k
1
vote
Program to automate skipping users on a chat website until the username meets certain conditions (v.2.0)
There are instructions you should not repeat inside the loop, and they should be written before it:
from bs4 import BeautifulSoup should be written much above in ...
- 3,392
1
vote
Page Object Model class structure in Selenium
Some lines may be simplified.
1) elementExists() already returns boolean, so use it directly:
...
- 328
1
vote
Accepted
Fill out webform with Selenium
Based on the comments I came to a final approach of fine-tuning the code.
Thank you all for your great advise, it was all really handy and usefull!
The following is done to improve the code:
More ...
1
vote
Fill out webform with Selenium
Some ideas for future improvements excluded those already mentioned by (@alecxe):
Avoid hard-coding path variables. It tends to change.
These lines are not very clear.
...
- 191
1
vote
Fill out webform with Selenium
Here are some things I would improve:
I think the method is overly loaded with code and is not readable. Consider applying Extract Method refactoring method and adding clarifying comments explaining ...
- 17.3k
1
vote
is it good practice to use relative path in xpath more than once
is it bad practice to use relative path(//) twice in a single xpath?
It would depend on the HTML source of the code - how many elements are there in the path between the nodes, how unique are the ...
- 17.3k
1
vote
Accepted
Email finding bot
Congratulations on putting your code up for review for the first time here. It takes some courage and humility to ask invite criticism of one's code.
Indentation
Standard Ruby intentation is two ...
- 3,284
Only top scored, non community-wiki answers of a minimum length are eligible