Checking whether customers in a dataframe have social media accounts

Question

I am using an API to check if customers have social media profiles.

Depending on whether they have phone or email or both there's a different search type.

There's a lot of conditions in this function, and the real dataframe has about 50 columns. So I'm just wondering if this is the most efficient way to go about it.

I'm aware that I'm applying this on a full row when I only need to work with a few in the df.

So I have two fake customer records here, and I'm trying to fill the social media columns with info returned from an API call:

import pandas as pd

df = pd.DataFrame(columns=['name','phone','email','facebook','foursquare','instagram','linkedin','skype','twitter'],index=range(0,2))

df['email'] = ['jim@email.com',pd.np.nan]
df['name'] = ['Jim Bob','Joe Bloggs']
df['phone'] = [pd.np.nan,'35543256']

print(df)

         name     phone          email facebook foursquare instagram linkedin  \
0     Jim Bob       NaN  jim@email.com      NaN        NaN       NaN      NaN   
1  Joe Bloggs  35543256            NaN      NaN        NaN       NaN      NaN   

  skype twitter  
0   NaN     NaN  
1   NaN     NaN 

Depending on the presence of phone/email, the function goes as follows (I believe I've commented my logic acceptably .. if something isn't clear please let me know)

# the columns from the df we want to fill
mycols = ['facebook','foursquare','instagram','linkedin','skype','twitter']

def checksocial(row):

    # if both phone and email are null
    if pd.isnull(row['phone']) and pd.isnull(row['email']):

        # do nothing
        # (analyzing and returning a whole row here, is this efficient?)
        return row

    # if there is no phone number but email is present
    elif pd.isnull(row['phone']) and pd.notnull(row['email']): 

        # use phone to search for social media
        # fake API response
        returned_results = ['facebook','foursquare','instagram']

        for socialmedia in returned_results:

            # if it's one of the social media profiles we are looking for
            if socialmedia in mycols:

                # add result to DF under same social media column
                row[socialmedia] = 'Found Social Media'
        # return updated row
        return row

    # if there is a phone number and email is empty
    elif pd.notnull(row['phone']) and pd.isnull(row['email']): 

        # use phone to search for social media
        # fake API response
        returned_results = ['facebook','linkedin','twitter']

        for socialmedia in returned_results:

            # if it's one of the social media profiles we are looking for
            if socialmedia in mycols:

                # add result to DF under same social media column
                row[socialmedia] = 'Found Social Media'

        # return updated row
        return row

    # repeat the same for when both email and phone are present

Applying the function:

df = df.apply(checksocial,axis=1)

print(df)

        name     phone          email            facebook  \
0     Jim Bob       NaN  jim@email.com  Found Social Media   
1  Joe Bloggs  35543256            NaN  Found Social Media   

           foursquare           instagram            linkedin  skype  \
0  Found Social Media  Found Social Media                 NaN    NaN   
1                 NaN                 NaN  Found Social Media    NaN   

              twitter  
0                 NaN  
1  Found Social Media  

It works fine, but the reason I'm asking for advice here is that the actual code I have is starting to become way too long. (There's parsing a json response that I didn't add here) and there are like 100,000 rows.

I have a lot of if statements, and I'm working with a full row and returning it.

Any advice on how to make this more cleaner/efficient?

Answer 1

EDIT: I've updated the function to check for phone and email.

Firstly, I think it's more readable and generally preferred to put a space after each comma in a list. PEP8 does not specifically mention this, but you will see that it is done in all their examples, i.e.:

my_list = [
    1, 2, 3,
    4, 5, 6,
]

Of course in the end that's up to you, it's just a guideline.

Your conditions only depend on phone and email, so there's no need to apply you function to the whole DataFrame. Also, I'm not sure whether your repeat the same for when both email and phone are present means that in that case you also use the phone-lookup API call or another one. If in case both are present you give one of the two preference, you use that as default. So your function could look something like this:

def checksocial(phone, email):
    returned_results = []
    if pd.notnull(phone) and pd.notnull(email):
        returned_results = ['skype', 'linkedin', 'twitter']  # "all" api call
    elif pd.notnull(phone) or pd.notnull(email):
        returned_results = (
            ['facebook', 'linkedin', 'twitter'] if pd.notnull(phone)
            else ['facebook', 'foursquare', 'instagram']
        )
    # I guess your returned result is a dict already, then you can skip this
    result = {social: 'Found Social Media' for social in returned_results}
    return pd.Series(result)

You then apply it to your two indicator columns and assign the result to the rest of your columns. Result with a slightly extended df that takes into account all possibilities:

columns = ['name', 'phone', 'email', 'facebook', 'foursquare', 'instagram',
           'linkedin', 'skype', 'twitter']
df = pd.DataFrame(columns=columns, index=range(0, 4))

df['email'] = ['jim@email.com', pd.np.nan, pd.np.nan, 'jane@email.com']
df['name'] = ['Jim Bob', 'Joe Bloggs', 'Chuck Norris', 'Jane Doe']
df['phone'] = [pd.np.nan, '35543256', pd.np.nan, '123456'

df.loc[:, mycols] = df.loc[:, ["phone", "email"]].apply(
    lambda x: checksocial(*x), axis=1)

print(df)

           name     phone           email            facebook  \
0       Jim Bob       NaN   jim@email.com  Found Social Media   
1    Joe Bloggs  35543256             NaN  Found Social Media   
2  Chuck Norris       NaN             NaN                 NaN   
3      Jane Doe    123456  jane@email.com                 NaN   

           foursquare           instagram            linkedin  \
0  Found Social Media  Found Social Media                 NaN   
1                 NaN                 NaN  Found Social Media   
2                 NaN                 NaN                 NaN   
3                 NaN                 NaN  Found Social Media   

                skype             twitter  
0                 NaN                 NaN  
1                 NaN  Found Social Media  
2                 NaN                 NaN  
3  Found Social Media  Found Social Media  

Answer 2

Few things to note here:

Columns:

email_results=['facebook','foursquare','instagram']
phone_results=['facebook','linkedin','twitter']

Conditions:

c1=df.phone.isna()&df.email.notna()
c2=df.phone.notna()&df.email.isna()

Method1

You can try and replace if else conditions with np.where , also take a look at np.select() if you have elif conditions:

df[email_results]=np.where(c1[:,None],'Found Social Media',df[email_results])
df[phone_results]=np.where(c2[:,None],'Found Social Media',df[phone_results])
print(df)

Method2

Or you can take a look at df.mask():

df[email_results]=df[email_results].mask(c1,'Found Social Media')
df[phone_results]=df[phone_results].mask(c2,'Found Social Media')
print(df)

---