3
\$\begingroup\$

I am using an API to check if customers have social media profiles.

Depending on whether they have phone or email or both there's a different search type.

There's a lot of conditions in this function, and the real dataframe has about 50 columns. So I'm just wondering if this is the most efficient way to go about it.

I'm aware that I'm applying this on a full row when I only need to work with a few in the df.

So I have two fake customer records here, and I'm trying to fill the social media columns with info returned from an API call:

import pandas as pd

df = pd.DataFrame(columns=['name','phone','email','facebook','foursquare','instagram','linkedin','skype','twitter'],index=range(0,2))

df['email'] = ['jim@email.com',pd.np.nan]
df['name'] = ['Jim Bob','Joe Bloggs']
df['phone'] = [pd.np.nan,'35543256']

print(df)

         name     phone          email facebook foursquare instagram linkedin  \
0     Jim Bob       NaN  jim@email.com      NaN        NaN       NaN      NaN   
1  Joe Bloggs  35543256            NaN      NaN        NaN       NaN      NaN   

  skype twitter  
0   NaN     NaN  
1   NaN     NaN 

Depending on the presence of phone/email, the function goes as follows (I believe I've commented my logic acceptably .. if something isn't clear please let me know)

# the columns from the df we want to fill
mycols = ['facebook','foursquare','instagram','linkedin','skype','twitter']

def checksocial(row):

    # if both phone and email are null
    if pd.isnull(row['phone']) and pd.isnull(row['email']):

        # do nothing
        # (analyzing and returning a whole row here, is this efficient?)
        return row

    # if there is no phone number but email is present
    elif pd.isnull(row['phone']) and pd.notnull(row['email']): 

        # use phone to search for social media
        # fake API response
        returned_results = ['facebook','foursquare','instagram']

        for socialmedia in returned_results:

            # if it's one of the social media profiles we are looking for
            if socialmedia in mycols:

                # add result to DF under same social media column
                row[socialmedia] = 'Found Social Media'
        # return updated row
        return row

    # if there is a phone number and email is empty
    elif pd.notnull(row['phone']) and pd.isnull(row['email']): 

        # use phone to search for social media
        # fake API response
        returned_results = ['facebook','linkedin','twitter']

        for socialmedia in returned_results:

            # if it's one of the social media profiles we are looking for
            if socialmedia in mycols:

                # add result to DF under same social media column
                row[socialmedia] = 'Found Social Media'

        # return updated row
        return row

    # repeat the same for when both email and phone are present

Applying the function:

df = df.apply(checksocial,axis=1)

print(df)

        name     phone          email            facebook  \
0     Jim Bob       NaN  jim@email.com  Found Social Media   
1  Joe Bloggs  35543256            NaN  Found Social Media   

           foursquare           instagram            linkedin  skype  \
0  Found Social Media  Found Social Media                 NaN    NaN   
1                 NaN                 NaN  Found Social Media    NaN   

              twitter  
0                 NaN  
1  Found Social Media  

It works fine, but the reason I'm asking for advice here is that the actual code I have is starting to become way too long. (There's parsing a json response that I didn't add here) and there are like 100,000 rows.

I have a lot of if statements, and I'm working with a full row and returning it.

Any advice on how to make this more cleaner/efficient?

\$\endgroup\$
  • \$\begingroup\$ What does repeat the same for when both email and phone are present mean exactly? Do you look up social media based on both? If so, what happens if the two lookups return different results? \$\endgroup\$ – iuvbio Jul 25 at 17:14
  • \$\begingroup\$ I have updated the code to explain that. Basically there's a different api call for each condition, depending on the presence of phone and email. not great but that's what I have to work with. \$\endgroup\$ – SCool Jul 26 at 10:23
  • \$\begingroup\$ Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers. \$\endgroup\$ – Zeta Jul 26 at 10:44
1
\$\begingroup\$

EDIT: I've updated the function to check for phone and email.

Firstly, I think it's more readable and generally preferred to put a space after each comma in a list. PEP8 does not specifically mention this, but you will see that it is done in all their examples, i.e.:

my_list = [
    1, 2, 3,
    4, 5, 6,
]

Of course in the end that's up to you, it's just a guideline.

Your conditions only depend on phone and email, so there's no need to apply you function to the whole DataFrame. Also, I'm not sure whether your repeat the same for when both email and phone are present means that in that case you also use the phone-lookup API call or another one. If in case both are present you give one of the two preference, you use that as default. So your function could look something like this:

def checksocial(phone, email):
    returned_results = []
    if pd.notnull(phone) and pd.notnull(email):
        returned_results = ['skype', 'linkedin', 'twitter']  # "all" api call
    elif pd.notnull(phone) or pd.notnull(email):
        returned_results = (
            ['facebook', 'linkedin', 'twitter'] if pd.notnull(phone)
            else ['facebook', 'foursquare', 'instagram']
        )
    # I guess your returned result is a dict already, then you can skip this
    result = {social: 'Found Social Media' for social in returned_results}
    return pd.Series(result)

You then apply it to your two indicator columns and assign the result to the rest of your columns. Result with a slightly extended df that takes into account all possibilities:

columns = ['name', 'phone', 'email', 'facebook', 'foursquare', 'instagram',
           'linkedin', 'skype', 'twitter']
df = pd.DataFrame(columns=columns, index=range(0, 4))

df['email'] = ['jim@email.com', pd.np.nan, pd.np.nan, 'jane@email.com']
df['name'] = ['Jim Bob', 'Joe Bloggs', 'Chuck Norris', 'Jane Doe']
df['phone'] = [pd.np.nan, '35543256', pd.np.nan, '123456'

df.loc[:, mycols] = df.loc[:, ["phone", "email"]].apply(
    lambda x: checksocial(*x), axis=1)

print(df)

           name     phone           email            facebook  \
0       Jim Bob       NaN   jim@email.com  Found Social Media   
1    Joe Bloggs  35543256             NaN  Found Social Media   
2  Chuck Norris       NaN             NaN                 NaN   
3      Jane Doe    123456  jane@email.com                 NaN   

           foursquare           instagram            linkedin  \
0  Found Social Media  Found Social Media                 NaN   
1                 NaN                 NaN  Found Social Media   
2                 NaN                 NaN                 NaN   
3                 NaN                 NaN  Found Social Media   

                skype             twitter  
0                 NaN                 NaN  
1                 NaN  Found Social Media  
2                 NaN                 NaN  
3  Found Social Media  Found Social Media  
\$\endgroup\$
  • \$\begingroup\$ Thank you. What's the asterisk for here? : checksocial(*x) I have also updated my question to answer your question: " Also, I'm not sure whether your repeat the same for when both email and phone are present means that in that case you also use the phone-lookup API call or another one." There's a separate API call for each condition. Not ideal but that's what I have to work with. The api call has a type argument for each condition. type: phone, type: email and type: all \$\endgroup\$ – SCool Jul 26 at 10:29
  • 1
    \$\begingroup\$ I see, that's good. Then I would recommend to first make a column indicating the type and then apply your api function to that type column. I will update my answer in a bit to reflect that. \$\endgroup\$ – iuvbio Jul 26 at 15:45
  • 1
    \$\begingroup\$ The asterisk unpacks the values it was given, so it will pass phone and email as separate values to checksocial. \$\endgroup\$ – iuvbio Jul 28 at 12:51
  • \$\begingroup\$ I've marked your response as the answer, thanks. Just one more question: for the function def checksocial, you create an empty list: returned_results = [] and then create returned_results again later: returned_results = ['skype', 'linkedin', 'twitter']. Why did you create the empty list first? Is it good practice to do this? \$\endgroup\$ – SCool Jul 30 at 11:38
  • 1
    \$\begingroup\$ If I didn't create the empty list, it could happen that returned_results is unassigned when it used to create the dict. Also, this way you don't need a separate condition for both phone and email being null, since it is the base case. As you write that you're parsing the json response, I guess that variable is already a dict and you don't need to create one. You'd probably want to initialise the varibable as None then. To your second question, elif (else if) if the first statement is true (phone and email), the second one is not evaluated. \$\endgroup\$ – iuvbio Jul 30 at 17:43
1
\$\begingroup\$

Few things to note here:

Columns:

email_results=['facebook','foursquare','instagram']
phone_results=['facebook','linkedin','twitter']

Conditions:

c1=df.phone.isna()&df.email.notna()
c2=df.phone.notna()&df.email.isna()

Method1

You can try and replace if else conditions with np.where , also take a look at np.select() if you have elif conditions:

df[email_results]=np.where(c1[:,None],'Found Social Media',df[email_results])
df[phone_results]=np.where(c2[:,None],'Found Social Media',df[phone_results])
print(df)

Method2

Or you can take a look at df.mask():

df[email_results]=df[email_results].mask(c1,'Found Social Media')
df[phone_results]=df[phone_results].mask(c2,'Found Social Media')
print(df)

enter image description here

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.