1
\$\begingroup\$

I have a dictionary that I am writing to a csv file, however I am only writing the values. I want to know if there is a faster way than using a for loop. I am using python 2.7 here is my code.

empty = ""

with open("parse_data.csv", "w") as f:
    for key, value in order_dic_keys.items():
        if value is None or value == "":
            empty += "" + "|"
        else:
            empty += str(value) + "|"
    f.write(empty[:])

As you can see I have a conditional statement to check the value. So I am unaware if there are other possibilities or if this is the best case scenario.

\$\endgroup\$
5
  • 4
    \$\begingroup\$ Is there any reason in particular you're stuck with Python 2.x? \$\endgroup\$
    – Mast
    Jan 24, 2022 at 17:36
  • 2
    \$\begingroup\$ What does your order_dic_keys look like, an ordinary dictionary? How big are we talking and will there be any nesting? If so, how far down? \$\endgroup\$
    – Mast
    Jan 24, 2022 at 17:37
  • \$\begingroup\$ What about DictWriter from module csv? Re-inventing the wheel? Tag! \$\endgroup\$
    – greybeard
    Jan 24, 2022 at 21:21
  • \$\begingroup\$ My order_dic_keys have 65 key and value pairs. As for the writer function from csv, I still need to do the conditional statement to check the values so I don't think that it would work here right? And I'm stuck on Python 2 because its what work uses. \$\endgroup\$
    – JThao
    Jan 24, 2022 at 21:23
  • \$\begingroup\$ @JThao No conditional statement is needed -- the csv package already handles the conversion of None into an empty string (as well as a bunch of other annoying details). You can see more details in my answer. \$\endgroup\$
    – josliber
    Jan 25, 2022 at 15:16

2 Answers 2

6
\$\begingroup\$

Writing to a csv file and grabbing all the values from a dictionary are two very common tasks in python, and so the language has built-in capabilities for both:

import csv

with open("parse_data.csv", "w") as f:
    writer = csv.writer(f, delimiter="|")
    writer.writerow(order_dic_keys.values())

This has a few nice properties compared to your code:

  • In your code, you needed to do string handling (e.g. concatenation), while this handles that for you.
  • This automatically handles annoying things to remember, like newlines at the end of your output (you actually didn't include one in your code)
  • This automatically handles data that has the "|" character in it, putting it in quotes so you don't interpret it as two separate data elements
  • This automatically handles data like None so you don't need to treat it as a special case, as you do in your code
  • It would be straightforward to add a header row -- you would just add writer.writerow(order_dic_keys.keys()) right after you create the writer object
\$\endgroup\$
4
  • \$\begingroup\$ Didn't know csv.writer already handled None like this. +1, obviously cleaner than my solution \$\endgroup\$ Jan 24, 2022 at 22:23
  • \$\begingroup\$ What if I need to close the file with a trailing "|" ? \$\endgroup\$
    – JThao
    Jan 26, 2022 at 15:56
  • \$\begingroup\$ @JThao I guess you could simply add an empty data element at the end of your data -- writer.writerow(order_dic_keys.values() + [""]) \$\endgroup\$
    – josliber
    Jan 26, 2022 at 16:28
  • \$\begingroup\$ This is perfect I didn't know how flexible it can be. Thank you \$\endgroup\$
    – JThao
    Jan 26, 2022 at 16:32
3
\$\begingroup\$

Without having any additional information, the following modifications will improve code style and should also improve performance (depending on size of data). On my machine, this approach provides a ~5x speed-up for a dictionary with 1 million random values and a ~15x speed-up for 5 million random values.

def value_to_str(value):
    return "" if value is None else str(value)

csv_data = "|".join(map(value_to_str, order_dic_keys.values()))

with open("parse_data.csv", "w") as f:
    f.write(csv_data)

Instead of manually stepping through the dictionary with a for-loop, using higher-order functions we take a functional approach to creating the csv-string. This should allow the interpreter to better optimize our code.


Notes:

  • Manual / Incremental string construction (string = string + string / string += string) is rather slow, try to avoid it when possible
  • empty += "" + "|", the empty string does nothing here --> empty += "|"
  • You don't need to and shouldn't create empty inside the open file context manager. The open file context manager should only contain code that absolutely has to be there
  • empty is only empty at initialization, it won't be empty at any other point during your program. It should not be named empty.
  • You don't need to copy the write-string empty[:] when writing it to the file, simply omit the [:]
  • As we're only using the dictionary values (not the keys), we can simply use dict.values instead of dict.items
\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.