Using Python to rename multiple csv files in Windows

Question

I need to rename a large list of csv files that are generated from 3 different servers. The files are produced with a date stamp as the extension, which I need to move in each file name to retain the date stamp.

The file name format before editing is as:

billing.pps-svr01.csv.2015-09-01

billing.pps-svr02.csv.2015-09-01

billing.pps-svr03.csv.2015-09-01

The file name format after editing should be:

billing.pps-svr01.2015-09-01.csv

billing.pps-svr02.2015-09-01.csv

billing.pps-svr03.2015-09-01.csv

My question is in regard to code efficiency and best practice. The following code seems to work in testing, however I'm very new to Python and programming in general and I'd like to know if there are any other ways to solve this problem that are more efficient...for example, could this be accomplished in a one-liner or something else? I should also note that I intend to incorporate this into a much larger script that parses these files after they've been renamed. Any feedback and/or suggestions would be great.

import os

os.chdir(r'C:\Users\Extract')

for filename in os.listdir('.'):
    if filename.startswith('billing.pps-svr'):
        name = filename.split('.')
        name = name+[name.pop(0-2)]
        new_name = '.'.join(name)
        os.rename(filename, new_name)

Answer 1

Here are my thoughts:

• You don't do any error handling if any of the file operation fails. Not to be recommended
• Your code will flip the latter two parts, all the time. It does not care if the filename has already been fixed or not!
• The code name + [name.pop(0-2)] troubles me. You are concatenating the name list with the popped value, but in order for this work you need for the popping to happen before the first part to be joined. Scary stuff...
• Here are some pythonic ways to do stuff with lists:
  • name[:-2] – Get everything but the two last elements
  • name[-2:] – Get only the last two elements
  • name[::-1] – Reverse the element list

Here is some coding displaying the flaw in your original rename code, and two options for how to handle it correctly.

for filename in (
                 'billing.pps-svr01.2014-09-01.csv',
                 'billing.pps-svr02.2014-09-01.csv',
                 'billing.pps-svr01.csv.2015-09-01',
                 'billing.pps-svr02.csv.2015-09-01',
                ):

    print('\nTesting {}:'.format(filename))

    name = filename.split('.')
    name = name + [name.pop(0-2)]
    new_name = '.'.join(name)

    print '    old rename: {} to {}'.format(filename, new_name)

    filename_parts = filename.split('.')
    first_part = filename_parts[:-2]
    last_part = filename_parts[-2:]

    if last_part[-1] != 'csv':
        new_name = '.'.join(first_part + last_part[::-1])
        print '    new rename: {} to {}'.format(filename, new_name)
    else:
        print '    no rename needed'

    if filename_parts[-2] == 'csv':
        new_name = '.'.join(filename_parts[:-2] + filename_parts[-2:][::-1])
        print '    alt rename: {} to {}'.format(filename, new_name)
    else:
        print '    no alternate rename needed'

The output from this are as follows:

Testing billing.pps-svr01.2014-09-01.csv:
    old rename: billing.pps-svr01.2014-09-01.csv to billing.pps-svr01.csv.2014-09-01
    no rename needed
    no alternate rename needed

Testing billing.pps-svr02.2014-09-01.csv:
    old rename: billing.pps-svr02.2014-09-01.csv to billing.pps-svr02.csv.2014-09-01
    no rename needed
    no alternate rename needed

Testing billing.pps-svr01.csv.2015-09-01:
    old rename: billing.pps-svr01.csv.2015-09-01 to billing.pps-svr01.2015-09-01.csv
    new rename: billing.pps-svr01.csv.2015-09-01 to billing.pps-svr01.2015-09-01.csv
    alt rename: billing.pps-svr01.csv.2015-09-01 to billing.pps-svr01.2015-09-01.csv

Testing billing.pps-svr02.csv.2015-09-01:
    old rename: billing.pps-svr02.csv.2015-09-01 to billing.pps-svr02.2015-09-01.csv
    new rename: billing.pps-svr02.csv.2015-09-01 to billing.pps-svr02.2015-09-01.csv
    alt rename: billing.pps-svr02.csv.2015-09-01 to billing.pps-svr02.2015-09-01.csv

Notice how the two first files would have gotten a wrongly rename use your original code.

Code refactor (added)

To accomodate for your question regarding building this into a larger script, and to give example of error handling, I've refactor your code into the following (using the tip from Janne Karila on using rsplit):

import os

def rename_csv_files(directory, required_start):
    """Rename files in <directory> starting with <required_start> to csv files

    Go to <directory> and read through all files, and for those
    starting with <required_start> and ending with something like 
    *.csv.YYYY-MM-DD and rename these to *.YYYY-MM-DD.
    """
    try:
       os.chdir(directory)
    except OSError, exception:
       print('IOError when changing directory - {}'.format(exception))
       return

    try:
        for filename in os.listdir('.'):
            if filename.startswith(required_start):

                base, ext, date = filename.rsplit('.', 2)
                new_filename = '.'.join((base, date, ext))
                if ext == 'csv' and not os.path.exists(new_filename):
                    try:
                        os.rename(filename, new_filename)
                        print 'Renamed: {}'.format(new_filename)

                    except OSError, exception:
                        print('Failed renaming file - dir: {}, original file:  {}, new file: {} - {}'.format(
                              directory, filename, new_filename, exception))

                elif ext != 'csv':
                    print('Skipped: {}'.format(filename))

                else:
                    print('Skipped: {} - Renamed version already exists'.format(filename))

    except OSError, exception:
        print('Failed traversing directory - dir: {} - {}'.format(directory, exception))

def main():
    rename_csv_files('./test_data', 'billing.pps-svr')

if __name__ == '__main__':
    main()

Running this script against the following test-data:

$ ls -1d test_data/* | sort -n
test_data/billing.pps-svr01.2014-09-01.csv
test_data/billing.pps-svr01.csv.2015-09-01
test_data/billing.pps-svr02.2014-09-01.csv
test_data/billing.pps-svr02.2015-09-01.csv
test_data/billing.pps-svr02.csv.2015-09-01
test_data/original_files.tar

Gives the following output:

Skipped: billing.pps-svr01.2014-09-01.csv
Renamed: billing.pps-svr01.2015-09-01.csv
Skipped: billing.pps-svr02.2014-09-01.csv
Skipped: billing.pps-svr02.2015-09-01.csv
Skipped: billing.pps-svr02.csv.2015-09-01 - Renamed version already exists

This code now handles error handling for at least the following cases:

• Directory not existing, or read or execution permission faults
• Errors when traversing directory or renaming files
• The logical error of renaming a file into an already existing file

Answer 2

The string manipulation becomes more readable using sequence unpacking:

base, ext, date = name.rsplit('.', 2)
new_name = '.'.join((base, date, ext))

Answer 3

I should also note that I intend to incorporate this into a much larger script that parses these files after they've been renamed

If so, you want modularity and customization possibilities, both of which can be achieved by wrapping your code in a function:

def sensible_name_for_this_renaming_process(directory, required_start):
    os.chdir(directory)

    for filename in os.listdir('.'):
        if filename.startswith(required_start):
            name = filename.split('.')
            name = name+[name.pop(0-2)]
            new_name = '.'.join(name)
    os.rename(filename, new_name)

Answer 4

It's slightly neater to avoid nesting by inverting the if condition at the start:

for filename in os.listdir('.'):
    if not filename.startswith('billing.pps-svr'):
        continue

    name = filename.split('.')
    name = name+[name.pop(0-2)]
    new_name = '.'.join(name)
    os.rename(filename, new_name)