I am using an API to check if customers have social media profiles.
Depending on whether they have phone or email or both there's a different search type.
There's a lot of conditions in this function, and the real dataframe has about 50 columns. So I'm just wondering if this is the most efficient way to go about it.
I'm aware that I'm applying this on a full row when I only need to work with a few in the df.
So I have two fake customer records here, and I'm trying to fill the social media columns with info returned from an API call:
import pandas as pd
df = pd.DataFrame(columns=['name','phone','email','facebook','foursquare','instagram','linkedin','skype','twitter'],index=range(0,2))
df['email'] = ['jim@email.com',pd.np.nan]
df['name'] = ['Jim Bob','Joe Bloggs']
df['phone'] = [pd.np.nan,'35543256']
print(df)
name phone email facebook foursquare instagram linkedin \
0 Jim Bob NaN jim@email.com NaN NaN NaN NaN
1 Joe Bloggs 35543256 NaN NaN NaN NaN NaN
skype twitter
0 NaN NaN
1 NaN NaN
Depending on the presence of phone/email, the function goes as follows (I believe I've commented my logic acceptably .. if something isn't clear please let me know)
# the columns from the df we want to fill
mycols = ['facebook','foursquare','instagram','linkedin','skype','twitter']
def checksocial(row):
# if both phone and email are null
if pd.isnull(row['phone']) and pd.isnull(row['email']):
# do nothing
# (analyzing and returning a whole row here, is this efficient?)
return row
# if there is no phone number but email is present
elif pd.isnull(row['phone']) and pd.notnull(row['email']):
# use phone to search for social media
# fake API response
returned_results = ['facebook','foursquare','instagram']
for socialmedia in returned_results:
# if it's one of the social media profiles we are looking for
if socialmedia in mycols:
# add result to DF under same social media column
row[socialmedia] = 'Found Social Media'
# return updated row
return row
# if there is a phone number and email is empty
elif pd.notnull(row['phone']) and pd.isnull(row['email']):
# use phone to search for social media
# fake API response
returned_results = ['facebook','linkedin','twitter']
for socialmedia in returned_results:
# if it's one of the social media profiles we are looking for
if socialmedia in mycols:
# add result to DF under same social media column
row[socialmedia] = 'Found Social Media'
# return updated row
return row
# repeat the same for when both email and phone are present
Applying the function:
df = df.apply(checksocial,axis=1)
print(df)
name phone email facebook \
0 Jim Bob NaN jim@email.com Found Social Media
1 Joe Bloggs 35543256 NaN Found Social Media
foursquare instagram linkedin skype \
0 Found Social Media Found Social Media NaN NaN
1 NaN NaN Found Social Media NaN
twitter
0 NaN
1 Found Social Media
It works fine, but the reason I'm asking for advice here is that the actual code I have is starting to become way too long. (There's parsing a json response that I didn't add here) and there are like 100,000 rows.
I have a lot of if
statements, and I'm working with a full row and returning it.
Any advice on how to make this more cleaner/efficient?
repeat the same for when both email and phone are present
mean exactly? Do you look up social media based on both? If so, what happens if the two lookups return different results? \$\endgroup\$ – iuvbio Jul 25 '19 at 17:14