New answers tagged web-scraping
2
My take, in short:
in programming there is nothing worse than silently swallowing exceptions :) And the fact that you are catching any exception, and not just those related to those specific operations, means than all kinds of errors will go unnoticed.
At the very least, the exceptions should be logged somewhere
be more specific and catch relevant ...
4
Before you can start thinking about doing fancy things like parallelizing this code, you should start by putting your code into functions that do one thing each.
def get_items(response):
# we only want the url before the query strings
# use a set to not have duplicate items
return {item["guid"].split("?")[0] for item in ...
Top 50 recent answers are included
