5
\$\begingroup\$

For practicing Python I wrote a script which scans the harddrive for large files and lists them in a report. The script takes the root path and then all the subfolders are scanned aswell. Also the minimum size for the files needs to be defined. Smaller files are not listed.

The code:

find_big_files.py

"""
Find big files on the harddrive and writes a report were the big files
are located
Supply root folder and desired size to find the files.
"""

import os

def size_with_biggest_unit(size: int) -> str:
    """
    Turns size in Bytes into KB, MB, GB or TB
    """
    unit: str = ""

    if size > (1024 ** 4):
        unit = "TB"
        size = int(round(size / 1024 ** 4))
    elif size > (1024 ** 3):
        unit = "GB"
        size = int(round(size / 1024 ** 3))
    elif size > (1024 ** 2):
        unit = "MB"
        size = int(round(size / 1024 ** 2))
    elif size > 1024 ** 1:
        unit = "KB"
        size = int(round(size / 1024))

    return str(size) + unit


def size_in_bytes(size: str) -> int:
    """
    Turns size in KB, MB, GB or TB into Byte
    """
    ret_size: int = 0
    if size.endswith("TB"):
        ret_size = int(size.strip("TB")) * (1024 ** 4)
    elif size.endswith("GB"):
        ret_size = int(size.strip("GB")) * (1024 ** 3)
    elif size.endswith("MB"):
        ret_size = int(size.strip("MB")) * (1024 ** 2)
    elif size.endswith("KB"):
        ret_size = int(size.strip("KB")) * 1024
    elif size.isdigit():
        ret_size = int(size)
    else:
        raise Exception(("Input size should be digit + TB/GB/MB/KB or digit."
                         "Size was: ") + size)
    return ret_size


def sort_by_size(big_files: list) -> list:
    """
    Sort dictionary with folder_name, filename and size by
    filesize in decreasind order
    """
    return sorted(big_files, key=lambda k: k['file_size'], reverse=True)


def write_report_of_big_files(big_files: list):
    """
    Write report in same folder as the script is excecuted.
    Report contains formated output of found big files
    """
    with open('big_files_report.txt', 'w') as file:
        for big_file  in big_files:
            file.write(size_with_biggest_unit(big_file['file_size'])
                       + '\t'
                       + big_file['filename']
                       + '\t'
                       + big_file['folder_name']
                       + '\n')


def find_big_files(root_folder: str, input_size: str):
    """
    Checks from all files in root and sub folder if they are exceeding
    a certain size
    """
    size = size_in_bytes(input_size)
    big_files: list = []

    for folder_name, subfolders, filenames in os.walk(root_folder):
        for filename in filenames:
            file_size = os.path.getsize(folder_name + '\\' + filename)

            if file_size > size:
                big_file: dict = {'folder_name': folder_name,
                                  'filename': filename,
                                  'file_size': file_size}
                big_files.append(big_file)

    sorted_big_files = sort_by_size(big_files)
    write_report_of_big_files(sorted_big_files)


find_big_files("E:\\", "100MB")

I checked the code with PyLint and MyPy.

PyLint still gives one warning in line 86 (in find_big_files ):

Unused variable 'subfolders' [W:unused-variable]

It is true I dont use the variable subfolders but I need to supply three variables in the for loop to use folder_name and filename which i get from os.walk(root_folder) or not?

Also I wonder if I used Type Annotations correctly. I used them here the first time and already found some bugs with MyPy (which are already fixed in the posted code).

Other than that:

Are there any other smells?

Is the code easy to follow?

Can anything be done easier?

Feel free to comment on anything suspicious you can find.

\$\endgroup\$
5
\$\begingroup\$
  • size_with_biggest_unit should be streamlined. Move the unit names into a list:

    unit_names = [ "", "KB", "MB", "TB" ]
    

and iterate downwards:

    for exponent in range (4, -1, -1):
        if size > 1024 ** exponent:
            unit_name = unit_names[exponent]
            return str(round(size / 1024 ** exponent)) + unit_name
    raise ImpossibleError

Same (almost same) applies to size_in_bytes.

  • Hardcoding \ as a path delimiter seriously impairs the portability. Prefer os.path.sep.

  • Instead of returning a list, consider turning it into an iterator.

  • Re Unused variable 'subfolders', a pythonic way to tell that the variable is truly unused is to call it _. I don't know if

    for folder_name, _, filenames in os.walk(root_folder):
    

    would pacify PyLint, but it would definitely make reviewer happier.

\$\endgroup\$
  • 1
    \$\begingroup\$ I would in general prefer os.path.join(folder_name, filename) over folder_name + os.path.sep + filename. \$\endgroup\$ – Graipher Dec 5 '18 at 11:39
  • \$\begingroup\$ shouldnt it be UNIT_NAMES = ( "", "KB", "MB", "GB", "TB"). Also what do you mean with raise ImpossibleError ? is it a placeholder for a raise Exception("Impossible Error")? \$\endgroup\$ – Sandro4912 Dec 5 '18 at 16:57
  • \$\begingroup\$ also i can confirm that _ for a not used variable is accepted by PyLint \$\endgroup\$ – Sandro4912 Dec 5 '18 at 17:09
  • \$\begingroup\$ Also i would appreciate an example for the returning of an iterator instead of the list. \$\endgroup\$ – Sandro4912 Dec 5 '18 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.