The following code asynchronously extract title from URLs (saved in
from bs4 import BeautifulSoup import grequests links = list() file1 = open("book-urls.txt", "r") Lines = file1.readlines() for line in Lines: links.append(line) file1.close() reqs = (grequests.get(link) for link in links) resp = grequests.map(reqs) for r in resp: try: soup = BeautifulSoup(r.text) print(soup.title.string) except: pass
The input is:
https://learning.oreilly.com/library/view/a-reviewers-handbook/9781118025635/ https://learning.oreilly.com/library/view/accounting-all-in-one-for/9781119453895/ https://learning.oreilly.com/library/view/business-valuation-for/9780470344019/ https://learning.oreilly.com/library/view/the-business-of/9780470444481/
The output is:
A Reviewer's Handbook to Business Valuation: Practical Guidance to the Use and Abuse of a Business Appraisal [Book] Accounting All-in-One For Dummies, with Online Practice, 2nd Edition [Book] Business Valuation For Dummies [Book] The Business of Value Investing: Six Essential Elements to Buying Companies Like Warren Buffett [Book]
grequests the best option or should I use something else?
Please give some suggestion on how I can make this code better.