Namedtuple and list to populate each record of a JSON file

Question

I had a JSON file in this format:

{
    "cols": [
        "Employee",
        "start_time"
    ],
    "data": [
        [
            "Serena",
            "Sat, 22 Aug 2015 14:06:03 -0700"
        ],
        [
            "Rhonda",
            "Sun, 25 Mar 2012 10:48:52 -0700"
        ],
        [
            "Fleur",
            "Mon, 16 Dec 2013 07:20:26 -0800"
        ],
        [
            "Giselle",
            "Sat, 19 Apr 2008 23:47:21 -0700"
        ],
        [
            "Jeanette",
            "Thu, 06 Nov 2008 23:02:44 -0800"
        ],
        [
            "Iliana",
            "Sun, 13 May 2007 14:22:08 -0700"
        ],
        [
            "Geraldine",
            "Tue, 24 Jul 2012 08:43:58 -0700"
        ],
        [
            "Tatiana",
            "Thu, 08 Oct 2009 07:56:25 -0700"
        ],
        [
            "Jessamine",
            "Wed, 14 Jun 2006 12:03:42 -0700"
        ]
        [
            "Emily",
            "Tue, 06 Jan 2015 04:51:06 -0800"
        ],
        [
            "Sydnee",
            "Mon, 16 Dec 2013 11:28:04 -0800"
        ],
        [
            "Zorita",
            "Wed, 16 Dec 2009 11:22:18 -0800"
        ]

    ]
}

The file has Employee name and start time fields. I wanted to get every employee's name and start time and find the total time they have been in the company (subtracting start time from current time).

I first loaded a JSON file using json.load() and then read each record in a loop and store it in a namedtuple. After that I added each namedtuple in a list.

import json
import pprint
import dateutil.parser
from collections import namedtuple
from datetime import datetime, timezone


file_name = "d:/a.json"
#Create a named tuple to store all data later,
# Total time is Current time - start_time
EmployeeData = namedtuple('EM', ["name", "start_time", "total_time"])

# Here I will store final list of all employee tuples
final_list = []


# Get string date as input and convert it to datetime object
def format_time(string_time):
    op_time = dateutil.parser.parse(string_time)
    return op_time


with open(file_name, "r") as data:
    json_data = json.load(data)
    for record in json_data["data"]:
        # Time in JSON file also has timezone so i have to use timezone.utc
        today = datetime.now(timezone.utc)
        # create date object from string date
        record[1] = format_time(record[1])
        # Find total number of days,
        tenure = (today - record[1]).days
        # create a tuple
        temp_tuple = EmployeeData(name=record[0], start_time = record[1], total_time = tenure)
        final_list.append(temp_tuple)

pprint.pprint(final_list)

Output:

 EM(name='Whitney', start_time=datetime.datetime(2015, 8, 7, 5, 37, 32, tzinfo=tzoffset(None, -25200)), total_time=1165),
 EM(name='Deirdre', start_time=datetime.datetime(2009, 8, 19, 15, 50, 27, tzinfo=tzoffset(None, -25200)), total_time=3343),
 EM(name='Alexandra', start_time=datetime.datetime(2007, 9, 5, 17, 31, 29, tzinfo=tzoffset(None, -25200)), total_time=4057),
 EM(name='Lila', start_time=datetime.datetime(2011, 8, 27, 8, 8, 47, tzinfo=tzoffset(None, -25200)), total_time=2606),
 EM(name='TaShya', start_time=datetime.datetime(2009, 1, 1, 18, 15, 1, tzinfo=tzoffset(None, -28800)), total_time=3573),
 EM(name='Kerry', start_time=datetime.datetime(2013, 6, 20, 13, 39, 30, tzinfo=tzoffset(None, -25200)), total_time=1942)]

I then sorted the record using:

sorted_time = sorted(final_list, key=lambda y: y.total_time)
print(sorted_time)

I would like to improve my code further since I will be dealing with a larger files soon. Is there any way to make it more efficient?

Answer 1

A small issue, that is probably no big deal for this program, but could cause problems over smaller timescales than days:

json_data = json.load(data)
for record in json_data["data"]:
    # Time in JSON file also has timezone so i have to use timezone.utc
    today = datetime.now(timezone.utc)

Having a different definition of "now" for each element in the file is unexpected. We can ensure consistency by assigning today once, outside the loop. (That slightly improves efficiency, too - but probably not by enough to measure!)