Questions tagged [scrapy]
Scrapy is a fast open-source high-level screen scraping and web crawling framework written in python, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
6
questions with no upvoted or accepted answers
3
votes
0
answers
61
views
Gather product information from the Apotheek Dedeyne web site
I'm learning Python Scrapy, and I'm scraping product information from the following website: https://apotheekdedeyne.be/
This project is for personal learning & ...
- 191
3
votes
0
answers
40
views
Scraping websites for presence of particular words
Im new to Python and Scrapy and Im trying to write a code that would find if a website contains a specific word in it. I need to scrapy many websites with different layouts so the search is a general ...
3
votes
0
answers
332
views
Microservice for scraping images with celery
I made a project, that scrapes images asynchronously and saves them in container. I have access to them through volume. Scrapy finds images on given web page.
Any tips will be good. But first I would ...
2
votes
0
answers
45
views
Use Scrapy to find duplicate images across pages
I'm trying to use scrapy to find image URLs that are used more than once on a website across all pages.
This is my spider:
...
- 121
2
votes
0
answers
180
views
Handling IndexError using lambda function within scrapy
I've written a script using python's scrapy library to parse some fields from craigslist. The spider I've created here is way normal than what usually gets considered ideal to be reviewed. However, I'...
- 2,471
1
vote
0
answers
154
views
Web scraping dynamic content
Hi I'm fairly new to coding and would appreciate some feedback on the code. This is for a site that has a dynamic login page and infinite scrolling.
I wanted to use scrapy not necessarily because it ...
- 19