I have 856k addresses of pattern:
660 1st Ave New York, NY 10016
(with 3 spaces between Ave and New York)
I need to parse and store parts as:
Street, City, State, Zip
I tried to split through regex but it was super slow, so I went traditional:
def splitAddress(address):
split = ' '
ad = address.strip()
try:
if(split in ad):
addr = ad.split(split)
if(len(addr)==2):
addr1 = addr[1].strip().split(",")
if len(addr1)==2:
addr2 = addr1[1].strip().split(" ")
if len(addr2)==2:
return addr[0], addr1[0], addr2[0], addr2[1]
else:
return "0","0","0","0"
else:
return "0","0","0","0"
else:
return "0","0","0","0"
else:
return "0","0","0","0"
except ValueError as e:
print(addr[0], addr1[0], addr2[0], addr2[1])
print(e)
I then got the parts as:
df['STREET'],df['CITY'], df['STATE'], df['ZIP'] = zip(*df['ADDRESS'].map(splitAddress))
But this is turning out to be pretty slow (it takes 18 mins for 856k addresses).