2
\$\begingroup\$

I'm retrieving e-mails from a collection of a few hundred *.json files using the following:

Python Script

import os, json

ordersDirectory = "C:\dev\orderArchive"
eMails = []
numUniqueEmails = 0
numEmails = 0

# json objects stored in multiple files, itterate over all of them
for filename in os.listdir(ordersDirectory):
    with open(ordersDirectory + '\\' + filename) as data_file:
        data = json.load(data_file)
        eMails.append(data["details"]["email"])
        numEmails +=1

# create and sort unique list of e-mails
uniqueEmails = sorted(list(set(eMails)))

print("Comma dilimited e-mails: \n")
for email in uniqueEmails:
    # print e-mails to console (comma delimited)
    numUniqueEmails += 1
    print(email + ",", end= " ")

print("\n\nNumber of unique e-mails is: " + str(numUniqueEmails) + " (originally " + str(numEmails) + ")")

Example JSON:

{  
   "details":{  
      "firstName":"Tester",
      "surname":"Testerman",
      "email":"[email protected]"
      },
   "orders":[  
      {  
         "name":"A test order",
         "quantity":20
      },
      {  
         "name":"Testable data",
         "quantity":20
      }]
}

Example output

Assuming that 30 files were processed and there were 3 unique e-mail addresses among them.

Comma dilimited e-mails:

[email protected], [email protected], [email protected],

Number of unique e-mails is: 3 (originally 30)

I know it's not very functional however I'm a C# developer primarily - I can get the job done in Python (and had to quickly for this task) but I'd love to know how to think more "pythonic". Any feedback or suggestions about style or performance are also appreciated.

\$\endgroup\$
2
  • 1
    \$\begingroup\$ Can you please add an example of the file you're extracting information from ? \$\endgroup\$ Jul 27, 2016 at 5:24
  • \$\begingroup\$ @Dex'ter Added an example. There's a lot more in the production data I can't post (for obv reasons) but the above should illustrate the important point of data and the use case. \$\endgroup\$
    – Michael A
    Jul 27, 2016 at 5:50

1 Answer 1

3
\$\begingroup\$

I'd recommend making use of a set, which stores only unique elements. That way you can just count insertions to get the total (with duplicates) and the unique ones are easy to retrieve.

Instead of iterating through unique emails and printing them with a comma and a space as a terminator (which gives you the odd comma and space at the end of the emails), make use of ", ".join().

Lastly, eMails should be emails in terms of punctuation, constants should be all caps, and camelCase is discouraged. For clarity, I changed the variable names to be more expressive.

Here is my complete solution:

import os, json

ORDERS_DIRECTORY = "C:\dev\orderArchive" # Reformatted constant.
unique_emails = set() # Holds only unique emails by design.
total_emails = 0 # Boring running total

# json objects stored in multiple files, itterate over all of them
for filename in os.listdir(ORDERS_DIRECTORY):
    with open(os.path.join(ORDERS_DIRECTORY, filename)) as data_file:
        data = json.load(data_file)
        unique_emails.add(data["details"]["email"]) # Make use of set here.
        total_emails += 1 # Corrected spacing here.

print("Comma dilimited e-mails: \n")
print(", ".join(unique_emails)) # Make use of join here.
print("\n\nNumber of unique e-mails is: {} (originally {})".format(len(unique_emails), total_emails))

On a side note, you may want to consider having some more error handling for the email field not being there, file not containing json etc. I'd also consider making the directory a program argument.

Update: fixed ORDERS_DIRECTORY consistency, use of os.path.join(), and str.format() per comments.

\$\endgroup\$
3
  • \$\begingroup\$ To add another comment to the above ones, I'd use os.path.join() instead of ordersDirectory + '\\' + filename \$\endgroup\$ Jul 27, 2016 at 14:42
  • \$\begingroup\$ @Graipher Done with both suggestions \$\endgroup\$
    – mleyfman
    Jul 27, 2016 at 19:22
  • \$\begingroup\$ @Dex'ter Done with your suggestion. \$\endgroup\$
    – mleyfman
    Jul 27, 2016 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.