Scraper for downloading and saving images from web page

Question

I've written some code using Python 3 to scrape movie names, links to the movie posters, and finally save the pictures on the local drive after downloading them from a web page.

I have used two functions to accomplish the whole task. I've tried my best to make the process clean. It is working great now. Any suggestion as to the betterment of this script will be very helpful for me. Thanks in advance. Here is the working code:

import requests
from lxml import html
import os

url = "https://www.yify-torrent.org/search/1080p/"

def ImageScraper(link):

    response = requests.session().get(link).text
    tree = html.fromstring(response)
    for title in tree.xpath('//div[@class="mv"]'):
        movie_title = title.findtext('.//h3/a')
        image_url = title.xpath('.//img/@src')[0]
        image_url = "https:" + image_url
        image_name = image_url.split('/')[-1]
        SavingImages(movie_title, image_name, image_url)

def SavingImages(movie_name, item_name, item_link):

    response = requests.session().get(item_link, stream = True)
    if response.status_code == 200:
        os.chdir(r"C:\Users\ar\Desktop\mth")
        with open(item_name, 'wb') as f:
            for chunk in response.iter_content(1024):
                f.write(chunk)

    print(movie_name, item_link)

ImageScraper(url)

Answer 1

I would focus on the following things specifically:

• variable and function naming:
  • use lower_case_with_underscores naming convention
  • what if we rename title to movie and movie_title to title - I think that would be a bit more descriptive
  • response should probably be named page_source since it is not a Response instance but already the text of the response
• use of spaces and line breaks:
  • according to PEP8 coding style, you should have 2 line breaks between the functions
  • when passing a keyword argument to a function, don't put spaces around the =
• code organization:
  • I would use a class to share a web-scraping session and have it parameterized with a url and a download directory. I think that would be more modular.

Improved code:

import os

import requests
from lxml import html


class ImageScraper:
    def __init__(self, url, download_path):
        self.url = url
        self.download_path = download_path

        self.session = requests.Session()

    def scrape_images(self):
        response = self.session.get(self.url).text

        tree = html.fromstring(response)
        for movie in tree.xpath('//div[@class="mv"]'):
            title = movie.findtext('.//h3/a')

            image_url = "https:" + movie.xpath('.//img/@src')[0]
            image_name = image_url.split('/')[-1]

            self.save_image(title, image_name, image_url)

    def save_image(self, movie_name, file_name, item_link):
        response = self.session.get(item_link, stream=True)

        if response.status_code == 200:
            with open(os.path.join(self.download_path, file_name), 'wb') as image_file:
                for chunk in response.iter_content(1024):
                    image_file.write(chunk)

        print(movie_name, file_name)


if __name__ == '__main__':
    scraper = ImageScraper(url="https://www.yify-torrent.org/search/1080p/",
                           download_path=r"C:\Users\ar\Desktop\mth")
    scraper.scrape_images()