I am running through a file and inserting elements one by one. The counties all contain specific county codes which are duplicated many times throughout the file. I am looking for a way to assign these codes to a specific county while ignoring duplicate county codes.
I have two versions I wrote below with runtimes:
getCodes()
def get_codes(data):
code_info = {}
for row in data:
county = row["county"]
code = int(float(row["code"]))
if code > 100000:
code = code/100
if county not in code_info:
code_info[county] = []
code_info[county].append(code)
for county in code_info:
code_info[county] = list(set(code_info[county]))
return code_info
get_codes2()
def get_codes2(data):
code_info = {}
for row in data:
county = row["county"]
code = int(float(row["code"]))
if code > 100000:
code = code/100
if county in code_info:
if not code in code_info[county]:
code_info[county].append(code)
else:
code_info[county] = []
code_info[county].append(code)
return code_info
county_data = csv.DictReader(open("county_file.txt"))
start = time.time()
county_codes = get_codes(county_data)
end = time.time()
print "run time: " + str(end-start)
county_data = csv.DictReader(open("county_file.txt"))
start = time.time()
county_codes = get_codes2(county_data)
end = time.time()
print "run time: " + str(end-start)
Also, it's probably obvious from this, but county codes that are greater than 100000 can have trailing zeroes accidentally added, so I'm removing them by dividing by 100. As another note, the int(float())
conversion is intentional. Sometimes the county codes are values such as "27.7" and need to be converted to "27", other times they are just basic int
s.
The runtimes on my system:
get_codes
: 9 secondsget_codes2
: 14 seconds
How can I improve this further for better performance?
defaultdict
, which might save some time with the "if county in code_info" checks. \$\endgroup\$