CSV parsing program that creates distinct header rows with transaction rows underneath

Question

My code reads in the data using DictReader, then creates a header row that contains my composite key (PEOPLE_ID, DON_DATE), and then adds various values that are distinct to each section. The output looks like this:

-01- PEOPLE_ID, DON_DATE, etc...
-02- dataline
-02- dataline
-01- ...
etc...

I'm looking to possibly simplify or streamline this code, and then could use advice on how to implement robust error-handling throughout. Here is my program:

#!/usr/bin/python
# pre_process.py
import csv
import sys


def main():
    infile = sys.argv[1]
    outfile = sys.argv[2]
    with open(infile, 'rbU') as in_obj:
        reader, fieldnames = open_reader(in_obj)
        reader = sorted(reader, key=lambda key: (key['PEOPLE_ID'], 
                                                 key['DON_DATE']))
        header_list = create_header_list(reader)
        master_dict = mapData(header_list, reader)
        writeData(master_dict, outfile, fieldnames)


def open_reader(file_obj):
    reader = csv.DictReader(file_obj, delimiter=',')
    return reader, reader.fieldnames


def create_header_list(dict_obj):
    p_id_list = []
    for row in dict_obj:
        if (row['PEOPLE_ID'], row['DON_DATE']) not in p_id_list:
            p_id_list.append((row['PEOPLE_ID'], row['DON_DATE']))
    return p_id_list


def mapData(header_list, dict_obj):
    master_dict = {}
    client_section_list = []
    for element in header_list:
        for row in dict_obj:
            if (row['PEOPLE_ID'], row['DON_DATE']) == element:
                client_section_list.append(row)
        element = list(element)
        element_list = [client_section_list[0]['DEDUCT_AMT'],
                    client_section_list[0]['ND_AMT'],
                    client_section_list[0]['DEDUCT_YTD'],
                    client_section_list[0]['NONDEDUCT_YTD']
                    ]
        try:
            element_list.append((float(client_section_list[0]['DEDUCT_YTD']) +
                                 float(client_section_list[0]['NONDEDUCT_YTD'])
                                 ))
        except ValueError:
            pass

        element.extend(element_list)
        element = tuple(element)
        master_dict[element] = client_section_list
        client_section_list = []
    return master_dict


def writeData(in_obj, outfile, in_fieldnames):
    with open(outfile, 'wb') as writer_outfile:
        writer = csv.writer(writer_outfile, delimiter=',')
        dict_writer = csv.DictWriter(writer_outfile,
                                     fieldnames=in_fieldnames,
                                     extrasaction='ignore')

        for k, v in in_obj.iteritems():
            writer_outfile.write(' -01- ')
            writer.writerow(k)
            for i, e in enumerate(v):
                writer_outfile.write(' -02- ')
                dict_writer.writerow(e)


def getReconTotals(infile):
    pass


if __name__ == '__main__':
    main()

Answer 1

Don't reuse names for multiple purposes

Before this line, reader is a DictReader, after this line it's a list:

    reader = sorted(reader, key=lambda key: (key['PEOPLE_ID'], 
                                             key['DON_DATE']))

This can be confusing. It would be better to name the result something else. And it gets worse: this new reader reader is passed to create_header_list and mapData as parameter named "dict_obj", which further adds to the confusion.

Simplify set creation

This function essentially creates a set:

def create_header_list(dict_obj):
    p_id_list = []
    for row in dict_obj:
        if (row['PEOPLE_ID'], row['DON_DATE']) not in p_id_list:
            p_id_list.append((row['PEOPLE_ID'], row['DON_DATE']))
    return p_id_list

The not in check is inefficient, because it's an \$O(n)\$ operation.

It would be simpler and more efficient to use a set:

def create_header_list(dict_obj):
    p_id_set = set()
    for row in dict_obj:
        p_id_set.add((row['PEOPLE_ID'], row['DON_DATE']))
    return p_id_set

Or even:

def create_header_list(dict_obj):
    return set([(row['PEOPLE_ID'], row['DON_DATE']) for row in dict_obj])

If the ordering of the elements is important, then instead of a set, you can use an OrderedDict, as suggested by this post.

Running Python scripts

Not all systems have Python at /use/bin/python. The recommended shebang for Python scripts:

#!/usr/bin/env python

Follow PEP8

PEP8 is the coding style guide for Python. Among other things, it recommends using snake_case for variable and function names. Several functions violate that.

Even if you disagree with a specific naming convention, it's a universal violation of good naming practices to mix two kinds of naming styles in the same program, such as create_header_list and mapData.