Simple Python script to sort and make json data unique

I'm retrieving e-mails from a collection of a few hundred *.json files using the following:

Python Script

import os, json

ordersDirectory = "C:\dev\orderArchive"
eMails = []
numUniqueEmails = 0
numEmails = 0

# json objects stored in multiple files, itterate over all of them
for filename in os.listdir(ordersDirectory):
with open(ordersDirectory + '\\' + filename) as data_file:
eMails.append(data["details"]["email"])
numEmails +=1

# create and sort unique list of e-mails
uniqueEmails = sorted(list(set(eMails)))

print("Comma dilimited e-mails: \n")
for email in uniqueEmails:
# print e-mails to console (comma delimited)
numUniqueEmails += 1
print(email + ",", end= " ")

print("\n\nNumber of unique e-mails is: " + str(numUniqueEmails) + " (originally " + str(numEmails) + ")")


Example JSON:

{
"details":{
"firstName":"Tester",
"surname":"Testerman",
"email":"test@test.com"
},
"orders":[
{
"name":"A test order",
"quantity":20
},
{
"name":"Testable data",
"quantity":20
}]
}


Example output

Assuming that 30 files were processed and there were 3 unique e-mail addresses among them.

Comma dilimited e-mails:

test@test.com, test2@test.com, emailhereagain@test.com,

Number of unique e-mails is: 3 (originally 30)

I know it's not very functional however I'm a C# developer primarily - I can get the job done in Python (and had to quickly for this task) but I'd love to know how to think more "pythonic". Any feedback or suggestions about style or performance are also appreciated.

• Can you please add an example of the file you're extracting information from ? – Grajdeanu Alex Jul 27 '16 at 5:24
• @Dex'ter Added an example. There's a lot more in the production data I can't post (for obv reasons) but the above should illustrate the important point of data and the use case. – Michael A Jul 27 '16 at 5:50

I'd recommend making use of a set, which stores only unique elements. That way you can just count insertions to get the total (with duplicates) and the unique ones are easy to retrieve.

Instead of iterating through unique emails and printing them with a comma and a space as a terminator (which gives you the odd comma and space at the end of the emails), make use of ", ".join().

Lastly, eMails should be emails in terms of punctuation, constants should be all caps, and camelCase is discouraged. For clarity, I changed the variable names to be more expressive.

Here is my complete solution:

import os, json

ORDERS_DIRECTORY = "C:\dev\orderArchive" # Reformatted constant.
unique_emails = set() # Holds only unique emails by design.
total_emails = 0 # Boring running total

# json objects stored in multiple files, itterate over all of them
for filename in os.listdir(ORDERS_DIRECTORY):
with open(os.path.join(ORDERS_DIRECTORY, filename)) as data_file:
unique_emails.add(data["details"]["email"]) # Make use of set here.
total_emails += 1 # Corrected spacing here.

print("Comma dilimited e-mails: \n")
print(", ".join(unique_emails)) # Make use of join here.
print("\n\nNumber of unique e-mails is: {} (originally {})".format(len(unique_emails), total_emails))


On a side note, you may want to consider having some more error handling for the email field not being there, file not containing json etc. I'd also consider making the directory a program argument.

Update: fixed ORDERS_DIRECTORY consistency, use of os.path.join(), and str.format() per comments.

• To add another comment to the above ones, I'd use os.path.join() instead of ordersDirectory + '\\' + filename – Grajdeanu Alex Jul 27 '16 at 14:42
• @Graipher Done with both suggestions – mleyfman Jul 27 '16 at 19:22
• @Dex'ter Done with your suggestion. – mleyfman Jul 27 '16 at 19:22