Questions tagged [scrapy]
Scrapy is a fast open-source high-level screen scraping and web crawling framework written in python, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
2
questions
6
votes
3
answers
701
views
Scraping a webpage copying with the logic of scrapy
Today, while coming across a tutorial made by ScrapingHub on Scrapy about how it usually deals with a webpage while scraping it's content. I could see that the same logic applied in Scrapy can be ...
- 2,471
4
votes
1
answer
948
views
Scraping a dynamic website with Scrapy (or Requests) and Selenium
I am trying to use Scrapy for one of the sites I've scraped before using Selenium over here.
Because the search field for this site is dynamically generated and requires the user to hover the cursor ...
- 417