1
\$\begingroup\$

my program requires a hash to be generated for files and folders for data integrity between transfers and file system events (if hash is different, file is not ready). For data integrity, I wish to use SHA-256 and for checking if a file is ready for transfer, I am using a custom hash that uses small parts of the start, middle and end and the size of the file. Please let me know if you need implementations, but I'm hoping that the interfaces are so self-explanatory that you don't need it.

I am somewhat new to dependency injection, so I wanted to ask you guys for opinions on any improvements, code smells, etc.

With these requirements, I have the following interfaces created:

The file system interface

class IFileSystem(ABC):
    @abstractmethod
    def open(
        self,
        path: str,
        mode: str = "r",
        buffering: int = -1,
        encoding: str | None = "utf-8",
        errors: str | None = None,
        newline: str = "",
    ) -> IO:
        """Opens a file at a given path on the file system

        Args:
            path: A path to a file on the filesystem.
            mode: The mode of opening, identical to the default python `open` method. Defaults to "r"
            buffering: Buffering policy (-1 to use default buffering, 0 to disable buffering, 1 to select line
                buffering, of any positive integer to indicate a buffer size).. Defaults to -1.
            encoding: Encoding for text files. Defaults to utf-8.
            errors: What to do with unicode. Defaults to None.
            newline: The newline format of the file. Defaults to "".

        Returns:
            IO: A working IO object for the file on the filesystem
        """

    @abstractmethod
    def list_dir(self, path: str) -> Iterable[FileSystemResource]:
        """Gets a list of all the resources in a directory on the file system. May be a file, directory

        Args:
            path: The path to the directory to list.

        Returns:
            List[FileSystemResourcer]: The file system resources in the directory
        """

    @abstractmethod
    def stat(self, path: str) -> FileSystemResource:
        """Retrieves the stat of a single resource at the given path

        Args:
            path: The path to the resource

        Returns:
            FileSystemResource: The info of the resource
        """

    @abstractmethod
    def copy(self, src: str, dest: str):
        pass

    @abstractmethod
    def remove(self, path: str):
        pass

    @abstractmethod
    def copy_dir(self, src: str, dest: str):
        pass

    @abstractmethod
    def remove_dir(self, path: str):
        pass

    @abstractmethod
    def walk(
        self, base_path: str
    ) -> Iterable[tuple[str, list[FileSystemResource], list[FileSystemResource]]]:
        """Walks the filesystem from the given path

        Args:
            path: The path to the first location to walk

        Yields:
            Iteratable[tuple[str, list[str], list[str]]]: Identical to python walk functionality
        """

class FileSystemResourceType(Enum):
    UNKNOWN = auto()
    DIRECTORY = auto()
    FILE = auto()
    CHARACTER = auto()
    BLOCK_SPECIAL_FILE = auto()
    FIFO = auto()
    SOCKET = auto()
    SYMLINK = auto()


@dataclass
class FileSystemResource:
    path: str
    name: str
    is_dir: bool
    size: int
    type: FileSystemResourceType
    accessed: Optional[datetime]
    created: Optional[datetime]
    metadata_changed: Optional[datetime]
    modified: Optional[datetime]

and the hashing interface

class IPathHasher(ABC):
    @abstractmethod
    def partial_hash_directory(self, path: str, fs: IFileSystem) -> str:
        """Hashes a directory starting from the given path. This hash will not guarantee content is complete, as it
        should sacrifice completeness for performance. Should be a reliable but not absolute indicator that a file has
        been completely copied

        Args:
            path: The path to the directory

        Returns:
            str: The hash
        """

    @abstractmethod
    def partial_hash(self, path: str, fs: IFileSystem) -> str:
        """Hashes a file object that is opened in binary mode. To open in binary mode, use `open(..., mode="rb")`.
        This hash is not suitable for ensuring content is complete, as it only uses a small amount of a file to hash it
        for performance reasons

        Args:
            file: The file object. Usually made with open or BytesIO()

        Returns:
            str: The hash of the file
        """

    @abstractmethod
    def complete_hash(self, path: str, fs: IFileSystem) -> str:
        """Hashes a file such that the entire content of the file is used to ensure that the entirety of 2 files are identical

        Args:
            file: The file to hash

        Returns:
            str: The hash
        """

    @abstractmethod
    def complete_hash_directory(self, path: str, fs: IFileSystem) -> str:
        """Hashes a directory using the complete hash algorithm. The hash should be completely unique to the directory.
        Collisions may lead to lost data.

        Args:
            path: The path to the directory

        Returns:
            str: The hash.
        """

Some potential changes I see that could be good but I am unsure about;

  • Use FileSystemResource objects instead of string paths everywhere except list_dir(...),stat(...) and the dest inputs to copy and copy_dir in the IFileSystem (or essentially anything that generates a FileSystemResource)
  • Same as above but for the IPathHasher
  • Separate the IPathHasher into an IPartialResourceHasher and ICompleteResourceHasher and change the interface to hash and hash_dir, then inject as needed. I am not sure how this works with IOC containers. Its simple enough in pure DI, but I honestly can't see how this works with a container. I'm only 1/4 way through a DI book and havent reached containers yet :p

I'm really keen for your responses because I have put a lot of thought into this program and I believe I finally have a fully fleshed out question to ask you guys that will give me a lot of insight for my own learning. Thanks.

\$\endgroup\$
1
  • 2
    \$\begingroup\$ Please show more than just your DI skeletons: include the implementation. \$\endgroup\$
    – Reinderien
    Jun 21 at 10:06

0

Browse other questions tagged or ask your own question.