12
votes
Accepted
Python script to scrape and parse the Stanford Encyclopedia of Philosophy
Add a hashbang at the top of your script since it's expected to be executable.
There is no function but I didn't think it was worth adding
Functions are worth adding: the visual and performance ...
- 65.4k
10
votes
Accepted
Repeatable HTTP requests in Python
If you are making many requests, it is worth it to have a look at requests.Session. This re-uses the connection to the server for subsequent requests (and also the ...
- 41.1k
10
votes
Python network manager class using exec()
sending_request is not a great name for an event; similar to your other signals, it should describe something that happened i.e. ...
- 65.4k
10
votes
Python network manager class using exec()
You don't need to use exec().
requests has a requests.request() function that takes the name ...
- 402
7
votes
Accepted
Web scraping Google Trends in Python
The idea behind your script is pretty cool and there's a couple ways to polish it up! While it certainly wouldn't cause a headache, I think that creating an entire class just to measure how long the ...
- 1,100
7
votes
Repeatable HTTP requests in Python
I'll start with the obvious:
while submissions > 0:
...
submissions -= 1
can be transformed into:
...
- 1,411
6
votes
Accepted
Python script to POST requests to a networking device
Disable hard certificate failures, fine (temporarily). But do not disable_warnings. They're there to remind you that you're doing a bad thing. You're a network ...
- 65.4k
6
votes
Accepted
Scraping web data with Python 3
I assume is_good_response is just checking for a 200 response code.
Merge is_good_response, ...
- 43.5k
5
votes
Accepted
Python: Handling different HTTP status codes using the requests library
I suggest you implement exponential back-off. For example, first try after x seconds and then twice as long as the last wait time. This is the common practice for retries. Recursion often looks ...
- 196
4
votes
Accepted
Python Web scraping
Naming
Variable names should be snake_case, and should represent what they are containing. I would also use req instead of <...
- 10.4k
4
votes
Python script to scrape and parse the Stanford Encyclopedia of Philosophy
General style
Use normal comments instead of triple quoted strings. Actually """ sep2epub """ doesn't really convey anything useful ...
- 1,184
4
votes
Accepted
Printing a JSON/HTTP response from a Cisco endpoint
Since this is the entire script,
it doesn't need to exist and you can just invoke curl directly; and
we can be a little more forgiving on exception-handling best ...
- 65.4k
4
votes
Python: Handling different HTTP status codes using the requests library
First, you should reduce to one call to request. You don't need to separate by method; just pass type to the method argument.
...
- 65.4k
3
votes
Accepted
Config Variables Initialization | Python
With no usage shown, no implementation for LookupDict shown and nothing other than your network type values, this is overdesigned and offers nothing beyond
...
- 65.4k
3
votes
Search for providers of TV Shows
PEP-8
Your code is mostly compliant with the Style Guide for Python Code. A few deviations should be corrected.
Function names should be in snake_case. So ...
- 34k
3
votes
Scrape webelements
Naming confusion
First of all there is something that is confusing in your code: bs4 does not actually represent an instance of BeautifulSoup. Pretty much any Python code that is based on ...
- 7,578
3
votes
Python Web scraping
I think you can even get rid of the regular expressions. I prefer to use the BS4 functions.
Instead of:
...
- 7,578
3
votes
Accepted
Single API call with authentication in Python
Instead of using:
esx_name = sys.argv[1]
prefer the argparse library which is more flexible and allows for variable argument positioning.
In ...
- 7,578
2
votes
Python requests downloading big files in slower speed than wget
By using time.sleep(1) on get_net_speed you are limiting your download speed to the block size per second, since your block has ...
2
votes
Accepted
Python program that gets basic information on a Github user
Just a note:
instead of using string.split() to check for spaces, you can directly check with in.
From:
...
2
votes
Python scraping tables
You're just looping on the rows, but not on the cells:
for row in all_tables[table].find_all("tr"):
Rather than using multiple ...
- 453
2
votes
PyCryptoScraper to scrape currency rates
Awesome job overall, it is a breeze to review a structured code like this.
Here are some of notes I've come up with:
optparse was deprecated long time ago, you ...
- 17.3k
2
votes
Accepted
Scraping webpage elements using Python
My take, in short:
in programming there is nothing worse than silently swallowing exceptions :) And the fact that you are catching any exception, and not just those related to those specific ...
- 7,578
2
votes
Accepted
Scrape webelements
You have not described the purpose of payload. If this is a JSON payload going to some other web service, Not found is a poor ...
- 65.4k
2
votes
Accepted
Run python File With get request flask API
To be honest this code looks pretty dangerous. exec is dangerous in its own right, and using it in conjunction with user-provided input makes an explosive ...
- 7,578
2
votes
Python script for Web scraping
When using Requests to fetch JSON content, it's usually more convenient to use the resonse.json() method instead of manually passing ...
- 3,737
2
votes
Accepted
Compare between two dictionaries
Most of this code should be deleted and replaced with API calls. Sign up for a free access token to increase your rate limit. I doubt threading will help.
Beyond that, the algorithm itself is ...
- 65.4k
2
votes
Accepted
python: requests large.zip -> unzip -> fix -> filter ->gunzip
I would probably just delete these constants:
GZ = ".gz"
READ = "r"
WRITE_BINARY = "wb"
They're pretty obvious.
These two parameters:...
- 65.4k
2
votes
Accepted
Create URL's, then download and print each RSS feed text
Your topics should be a tuple and not a list, and if you want they can be moved to a global constant.
Add PEP484 type hints.
I don't think that feedparser is able ...
- 65.4k
2
votes
Python: Handling different HTTP status codes using the requests library
1: Is a try/except block like I have an acceptable way of handling error codes?
By default, requests library does not raise exceptions based on HTTP status codes. ...
Only top scored, non community-wiki answers of a minimum length are eligible