my program requires a hash to be generated for files and folders for data integrity between transfers and file system events (if hash is different, file is not ready). For data integrity, I wish to use SHA-256 and for checking if a file is ready for transfer, I am using a custom hash that uses small parts of the start, middle and end and the size of the file. Please let me know if you need implementations, but I'm hoping that the interfaces are so self-explanatory that you don't need it.
I am somewhat new to dependency injection, so I wanted to ask you guys for opinions on any improvements, code smells, etc.
With these requirements, I have the following interfaces created:
The file system interface
class IFileSystem(ABC): @abstractmethod def open( self, path: str, mode: str = "r", buffering: int = -1, encoding: str | None = "utf-8", errors: str | None = None, newline: str = "", ) -> IO: """Opens a file at a given path on the file system Args: path: A path to a file on the filesystem. mode: The mode of opening, identical to the default python `open` method. Defaults to "r" buffering: Buffering policy (-1 to use default buffering, 0 to disable buffering, 1 to select line buffering, of any positive integer to indicate a buffer size).. Defaults to -1. encoding: Encoding for text files. Defaults to utf-8. errors: What to do with unicode. Defaults to None. newline: The newline format of the file. Defaults to "". Returns: IO: A working IO object for the file on the filesystem """ @abstractmethod def list_dir(self, path: str) -> Iterable[FileSystemResource]: """Gets a list of all the resources in a directory on the file system. May be a file, directory Args: path: The path to the directory to list. Returns: List[FileSystemResourcer]: The file system resources in the directory """ @abstractmethod def stat(self, path: str) -> FileSystemResource: """Retrieves the stat of a single resource at the given path Args: path: The path to the resource Returns: FileSystemResource: The info of the resource """ @abstractmethod def copy(self, src: str, dest: str): pass @abstractmethod def remove(self, path: str): pass @abstractmethod def copy_dir(self, src: str, dest: str): pass @abstractmethod def remove_dir(self, path: str): pass @abstractmethod def walk( self, base_path: str ) -> Iterable[tuple[str, list[FileSystemResource], list[FileSystemResource]]]: """Walks the filesystem from the given path Args: path: The path to the first location to walk Yields: Iteratable[tuple[str, list[str], list[str]]]: Identical to python walk functionality """ class FileSystemResourceType(Enum): UNKNOWN = auto() DIRECTORY = auto() FILE = auto() CHARACTER = auto() BLOCK_SPECIAL_FILE = auto() FIFO = auto() SOCKET = auto() SYMLINK = auto() @dataclass class FileSystemResource: path: str name: str is_dir: bool size: int type: FileSystemResourceType accessed: Optional[datetime] created: Optional[datetime] metadata_changed: Optional[datetime] modified: Optional[datetime]
and the hashing interface
class IPathHasher(ABC): @abstractmethod def partial_hash_directory(self, path: str, fs: IFileSystem) -> str: """Hashes a directory starting from the given path. This hash will not guarantee content is complete, as it should sacrifice completeness for performance. Should be a reliable but not absolute indicator that a file has been completely copied Args: path: The path to the directory Returns: str: The hash """ @abstractmethod def partial_hash(self, path: str, fs: IFileSystem) -> str: """Hashes a file object that is opened in binary mode. To open in binary mode, use `open(..., mode="rb")`. This hash is not suitable for ensuring content is complete, as it only uses a small amount of a file to hash it for performance reasons Args: file: The file object. Usually made with open or BytesIO() Returns: str: The hash of the file """ @abstractmethod def complete_hash(self, path: str, fs: IFileSystem) -> str: """Hashes a file such that the entire content of the file is used to ensure that the entirety of 2 files are identical Args: file: The file to hash Returns: str: The hash """ @abstractmethod def complete_hash_directory(self, path: str, fs: IFileSystem) -> str: """Hashes a directory using the complete hash algorithm. The hash should be completely unique to the directory. Collisions may lead to lost data. Args: path: The path to the directory Returns: str: The hash. """
Some potential changes I see that could be good but I am unsure about;
- Use FileSystemResource objects instead of string paths everywhere except
copy_dirin the IFileSystem (or essentially anything that generates a FileSystemResource)
- Same as above but for the
- Separate the
ICompleteResourceHasherand change the interface to
hash_dir, then inject as needed. I am not sure how this works with IOC containers. Its simple enough in pure DI, but I honestly can't see how this works with a container. I'm only 1/4 way through a DI book and havent reached containers yet :p
I'm really keen for your responses because I have put a lot of thought into this program and I believe I finally have a fully fleshed out question to ask you guys that will give me a lot of insight for my own learning. Thanks.