0
\$\begingroup\$

I am trying to create a dictionary with a file containing text based on a matched pattern. Lines containing key_str should become keys and subsequent lines not matching key_str should become values and get associated with keys in the dictionary. so i have below code working: But I need help and getting the

  1. best practices and design pattern in my case
  2. Performance in case my file is in GB
  3. Alternate/different approaches without using any modules but just raw logic. don't want to use collections for example. for using python module I can ask a different question but here I just want to get to raw logic approaches.

File: file2dict.result-soa1

ml1
/var
/home

cpuml2
/var
/home

Output

my_dict: {ml1: ['/var','/home'], cpuml2: ['/var','/home']}

Code:

import os
homedir = os.environ.get('HOME')

key_str = "ml"
my_dict = {}
val_list = []
key = ''
with open(homedir + '/backup/file2dict.result-soa1') as file2dict:
    for line in file2dict:
        words = line.split()
        for aWord in words:
            if key_str in aWord:
                if key:
                    my_dict[key] = val_list
                    val_list = []
                    key = aWord
                else:
                    key = aWord
            else:
                val_list.append(aWord)
    my_dict[key] = val_list
print(my_dict)
\$\endgroup\$
0
\$\begingroup\$

Your code is off to a good start, but here are some suggestions either for modest improvements or ways to simplify and clarify things.

The built-in pathlib library is often the easiest and most robust way to work with file paths.

Scripts with hard-coded file paths tend to be inflexible. For example, to review your code I had to edit it right out of the gate, because my computer lacks your directory structure. During your own development and debugging of a script, you might not want to operate on the real file every time (using a tiny replica is often easier). All of which is to say that you are better off using default paths with the option of command-line overrides. This adjustment also positions you well for the future: if the needs of a script increase, you can easily add support for more command-line arguments or options.

I would encourage you to embrace the time-tested best-practice of putting nearly all substantive code inside of function or class definitions. It's an easy change to make and the greater isolation tends to increase flexibility. For example, in the suggested edits below, which still stick pretty closely to your original code, the algorithmic complexity of parsing has been extracted into a narrowly tailored function that knows nothing about the program's larger concerns. It doesn't have to know about paths, file reading, etc. Instead, it takes any iterable containing lines of relevant data. That makes it easier to debug and test, because you can call it separately with just a list of strings.

The edits below also provide a short illustration of how even fairly brief comments can add context and help the reader understand both purpose and the general logic. Some of the comments are like organizational sign posts: for example, "Setup" or "Parse". They make it easier to scan code visually to get your bearings. Other comments can help a reader by explain the gist of the algorithm intuitively (for example, the two comments in the inner loop). Notice that the comments do not mimic the code; rather, they help to organize it and throw additional light on it.

At a minimum, I would encourage you to adopt consistent variable naming conventions. Sometimes you follow the practice of appending data-type suffixes to names; other times not. That's not a practice I've ever found valuable, but opinions differ. Naming variables, functions, and classes is difficult because there are often competing priorities: clarity, brevity, ease of typing, ease of reading, need to compare/contrast similar things, and so forth. In the edits below, some of the names below are conventional. For example, unless I need to have multiple files open at once, I always use fh for "file handle". Same thing for args in a main() function. Other names are fairly explicit and human-readable: default, file_name, file_path, line, and word are examples in this bucket. And other names are less strong: for example, data and vals. In some contexts, a generic name is appropriate (for example, a general-purpose function that performs a general operation on some input). In your case, I suspect that more substantive names exist, but only you know them.

from sys import argv
from pathlib import Path

def main(args):
    # Get file path to the input data file.
    default = 'backup/file2dict.result-soa1'
    file_name = args[0] if args else default
    file_path = Path.home() / file_name

    # Parse.
    with open(file_path) as fh:
        data = parse_data(fh)

    # Report.
    print(data)

def parse_data(lines):
    # Setup.
    data = {}
    vals = []
    key = None
    marker = 'ml'

    # Parse.
    for line in lines:
        for word in line.split():
            if marker in word:
                # Store current VALS; and setup new KEY and VALS.
                if key:
                    data[key] = vals
                    vals = []
                key = word
            else:
                # Or just store with current VALS.
                vals.append(word)

    # Wrap-up and return.
    data[key] = vals
    return data

if __name__ == '__main__':
    main(argv[1:])
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.