10
\$\begingroup\$

This code that I wrote is supposed to read/write a pipe-delimited file line by line to a new file with some simple text manipulation. (It also adds two new columns) and publishes a "Status Update" ever 100,000 lines to keep me updated on how close it is to completion.

I previously posted this code on StackOverflow to get help with incrementing, and someone mentioned that it would be faster if I did not open the second text file, but being extremely new at Python, I do not understand how to do that without potentially breaking the code.

counter=1
for line in open(r"C:\Path\name.txt"):
    spline = line.split("|")
    if counter==1:
        with open(r"C:\PATH\2019.txt",'a') as NewFile:
            spline.insert(23,"Column A")
            spline.insert(23,"Column B")
            s="|"
            newline=s.join(spline)
            NewFile.write(newline)

    elif counter > 1 and not spline[22]=="0.00":
        spline.insert(23,"")
        spline.insert(23,"")
        gl=spline[0]
        gl=gl.strip()
        if gl[0]=="-": gl="000" + gl
        gl=gl.upper()
        spline[0]=gl

        if gl[:3]=="000": spline[24]="Incorrect"

        s="|"
        newline=s.join(spline)

        with open(r"C:\PATH\PythonWrittenData.txt",'a') as NewFile:
            NewFile.write(newline)

    counter+=1          
    if counter%100000==0: print("Status Update: \n", "{:,}".format(counter)) 
\$\endgroup\$
  • \$\begingroup\$ Can you edit your question to explain what the loop is supposed to do? \$\endgroup\$ – Linny Apr 1 at 14:28
  • \$\begingroup\$ @DavidWhite Added. For quick reference: It reads one text file line by line, performs string manipulation, and then writes it into a new file line by line. \$\endgroup\$ – Emily Alden Apr 1 at 14:56
7
\$\begingroup\$

A nice trick you can use in python is to open two (or more) files at once in one line. This is done with something like:

with open('file_one.txt', 'r') as file_one, open('file_two.txt', 'r') as file_two:
    for line in file_one:
        ...
    for line in file_two:
        ...

This is a very common way of reading from one file and writing to another without continually opening and closing one of them.

Currently, you're opening and closing the files with each iteration of the loop. Your program loops through the lines in name.txt, checks an if / elif condition, then if either are satisfied, a file is opened, written to, then closed again with every iteration of the loop.

Simply by opening both files at the same time you can stop opening and closing them repeatedly.

For more info on the with statement and other context managers, see here.


Another small improvement can be made. At the moment, you check the first if condition every time, but you know it will only actually evaluate to True once. it would be better to remove that check and just always perform that block once. Assign counter after the first block (after where if counter == 1 currently is) then replace the elif statement with a while loop.


It would be worth getting familiar with PEP8 if you're going to use Python a lot in the future. It's a standard style guide and will help with the readability of your code (for you and others). Just small stuff like new lines after colons or spaces either side of variable declarations / comparisons.


If you include an example file and desired output, there may be more I can help with.

\$\endgroup\$
  • \$\begingroup\$ I decided to accept this answer as it has the information most pertinent to my skill level and it worked. (I couldn't get the iterative answer to produce the expected results). Thank you very much. I am modifying my code so that it will be PEP8 compliant. \$\endgroup\$ – Emily Alden Apr 1 at 21:07
8
\$\begingroup\$

Here is another way to organize your code. Instead of an if within the loop, use iterators more explicitly. Concretely:

with open(r"C:\Path\name.txt") as source:
    lines = iter(source)

    # first line
    first_line =  next(lines)
    with open(r"C:\PATH\2019.txt") as summary:
        # ... omitted ...

    # remaining lines
    with open(r"C:\PATH\PythonWrittenData.txt", 'a') as dest:
        for counter, line in enumerate(lines, start=1):
            # ... omitted ...

I have also used enumerate to update counter and line simultaneously.

The other answer has some more tips on writing good python code. But as far as structuring the opening and closing of files, as well as the main loop, this approach should get you started.

\$\endgroup\$
  • \$\begingroup\$ I'm not sure I understand the reasoning behind first_line=next(lines) could you explain how it should be used? \$\endgroup\$ – Emily Alden Apr 1 at 18:22
  • \$\begingroup\$ @EmilyAlden Why accept an answer when you don’t understand why that answer works? \$\endgroup\$ – Linny Apr 1 at 19:11
  • \$\begingroup\$ @DavidWhite: Honestly I bounced between which answer to accept. I understand the use of iter and enumerate here which are the main parts of the answer. The line I do not understand applies only once and I believe I can write around that line and get the results I want. (Still working on it though). \$\endgroup\$ – Emily Alden Apr 1 at 19:36
  • 1
    \$\begingroup\$ @EmilyAlden first_line = next(iterator) is equivalent to for first_line in iterator: break \$\endgroup\$ – wizzwizz4 Apr 1 at 19:48
  • \$\begingroup\$ @wizzwizz4 Thank you. \$\endgroup\$ – Emily Alden Apr 1 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.