Questions tagged [scrapy]

Scrapy is a fast open-source high-level screen scraping and web crawling framework written in python, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

6 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers

Gather product information from the Apotheek Dedeyne web site

I'm learning Python Scrapy, and I'm scraping product information from the following website: This project is for personal learning & ...
Seraph776's user avatar
  • 191
3 votes
0 answers

Scraping websites for presence of particular words

Im new to Python and Scrapy and Im trying to write a code that would find if a website contains a specific word in it. I need to scrapy many websites with different layouts so the search is a general ...
Andris444444444444's user avatar
3 votes
0 answers

Microservice for scraping images with celery

I made a project, that scrapes images asynchronously and saves them in container. I have access to them through volume. Scrapy finds images on given web page. Any tips will be good. But first I would ...
Grzegorz Krug's user avatar
2 votes
0 answers

Use Scrapy to find duplicate images across pages

I'm trying to use scrapy to find image URLs that are used more than once on a website across all pages. This is my spider: ...
W2a's user avatar
  • 121
2 votes
0 answers

Handling IndexError using lambda function within scrapy

I've written a script using python's scrapy library to parse some fields from craigslist. The spider I've created here is way normal than what usually gets considered ideal to be reviewed. However, I'...
SIM's user avatar
  • 2,471
1 vote
0 answers

Web scraping dynamic content

Hi I'm fairly new to coding and would appreciate some feedback on the code. This is for a site that has a dynamic login page and infinite scrolling. I wanted to use scrapy not necessarily because it ...
Aaron S's user avatar
  • 19