Is there a faster way to write dictionary values to csv?

Question

I have a dictionary that I am writing to a csv file, however I am only writing the values. I want to know if there is a faster way than using a for loop. I am using python 2.7 here is my code.

empty = ""

with open("parse_data.csv", "w") as f:
    for key, value in order_dic_keys.items():
        if value is None or value == "":
            empty += "" + "|"
        else:
            empty += str(value) + "|"
    f.write(empty[:])

As you can see I have a conditional statement to check the value. So I am unaware if there are other possibilities or if this is the best case scenario.

Answer 1

Writing to a csv file and grabbing all the values from a dictionary are two very common tasks in python, and so the language has built-in capabilities for both:

import csv

with open("parse_data.csv", "w") as f:
    writer = csv.writer(f, delimiter="|")
    writer.writerow(order_dic_keys.values())

This has a few nice properties compared to your code:

• In your code, you needed to do string handling (e.g. concatenation), while this handles that for you.
• This automatically handles annoying things to remember, like newlines at the end of your output (you actually didn't include one in your code)
• This automatically handles data that has the "|" character in it, putting it in quotes so you don't interpret it as two separate data elements
• This automatically handles data like None so you don't need to treat it as a special case, as you do in your code
• It would be straightforward to add a header row -- you would just add writer.writerow(order_dic_keys.keys()) right after you create the writer object

Answer 2

Without having any additional information, the following modifications will improve code style and should also improve performance (depending on size of data). On my machine, this approach provides a ~5x speed-up for a dictionary with 1 million random values and a ~15x speed-up for 5 million random values.

def value_to_str(value):
    return "" if value is None else str(value)

csv_data = "|".join(map(value_to_str, order_dic_keys.values()))

with open("parse_data.csv", "w") as f:
    f.write(csv_data)

Instead of manually stepping through the dictionary with a for-loop, using higher-order functions we take a functional approach to creating the csv-string. This should allow the interpreter to better optimize our code.

---

Notes:

• Manual / Incremental string construction (string = string + string / string += string) is rather slow, try to avoid it when possible
• empty += "" + "|", the empty string does nothing here --> empty += "|"
• You don't need to and shouldn't create empty inside the open file context manager. The open file context manager should only contain code that absolutely has to be there
• empty is only empty at initialization, it won't be empty at any other point during your program. It should not be named empty.
• You don't need to copy the write-string empty[:] when writing it to the file, simply omit the [:]
• As we're only using the dictionary values (not the keys), we can simply use dict.values instead of dict.items