Questions tagged [scrapy]

Scrapy is a fast open-source high-level screen scraping and web crawling framework written in python, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Filter by
Sorted by
Tagged with
6 votes
3 answers
701 views

Scraping a webpage copying with the logic of scrapy

Today, while coming across a tutorial made by ScrapingHub on Scrapy about how it usually deals with a webpage while scraping it's content. I could see that the same logic applied in Scrapy can be ...
SIM's user avatar
  • 2,471
4 votes
1 answer
948 views

Scraping a dynamic website with Scrapy (or Requests) and Selenium

I am trying to use Scrapy for one of the sites I've scraped before using Selenium over here. Because the search field for this site is dynamically generated and requires the user to hover the cursor ...
Sati's user avatar
  • 417