I have 2 pandas dataframes. I want to do a find and replace between 2 dataframes. In the df_find dataframe, in the current_title column, i want to search in every row for any occurrence of values from 'keywrod' column in the df_replace dataframe and if found replace it with corresponding value from 'keywordlength' column.

I have been able to get rid of the loop for df_find dataframe since i need to iterate over every row in this dataframe by using str.replace which is a vectorized form of replace function.

Performance matters in my case, as both the dataframes run into GB's. So, i want to get rid of the loop for df_replace here and use any other efficient way of iterating through all rows of df_replace dataframe.

import pandas as pd
df_find = pd.read_csv("input_find.csv")
df_replace = pd.read_csv("input_replace.csv")

#replace
for i,j in zip(df_replace.keyword,df_replace.keywordLength):
    df_find.current_title=df_find.current_title.str.replace(i,j,case=False)

df_replace This dataframe has the data we need for find and replace

keyword       keywordLength
IT Manager    ##10##
Sales Manager ##13##
IT Analyst    ##12##
Store Manager ##13##

df_find is where we need to do the transformation.

Before executing find and replace code:

current_title
I have been working here as a store manager since after I passed from college
I am sales manager and primarily work in the ASEAN region. My primary rolw is to bring new customers.
I initially joined as a IT analyst and because of my sheer drive and dedication, I was promoted to IT manager position within 3 years

After executing find and replace through above code

current_title
I have been working here as a ##13## since after I passed from college
I am ##13## and primarily work in the ASEAN region. My primary rolw is to bring new customers.
I initially joined as a ##12## and because of my sheer drive and dedication, I was promoted to ##10## position within 3 years

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.