Efficient way to add a colon before and after every word in strings inside a Series

Question

I was working with twitter data, and extracted every emoji from the data. But, when I passed that data through CountVectorizer, colons where substracted from the strings. So, the string emoji :ok_hand: :thumbs_up: turned into ok_hand thumbs_up. I wanted to re-add those colons so then I could emojize them back. I managed to do that, but I'm quite sure my method is very inefficient. The emojis are the indexes of a coefficients DataFrame, like this:

    index              coef

ok_hand thumbs_up      0.4
    airplane           0.2

So what I did was this:

to_emojize=pd.Series(coef_mat_emoji.index)
to_emojize=to_emojize.apply(lambda x: x.split())
to_emojize=to_emojize.apply(lambda x:[':'+i+':' for i in x])
to_emojize=to_emojize.apply(lambda x: emoji.emojize(x, use_aliases=True))
coef_mat_emoji.index=to_emojize

Is there a better way to do this?

Answer 1

Both pandas.Series and pandas.Index have vectorized string additions. You can just do:

to_emojize = pd.Series(":" + coef_mat_emoji.index + ":")
coef_mat_emoji.index = to_emojize.apply(emoji.emojize, use_aliases=True)

Note that pandas.Series.apply passes any additional keyword arguments along to the function, so there is no need for the lambda here at all.

This will create an intermediate series from the first addition, which might not be the most memory efficient way to do this. But it is the easiest and most readable, so unless you run out of memory with this, this is what I would use.

Alternatively you could put it all into one apply call (Python 3.6+ for f-strings):

coef_mat_emoji.index = pd.Series(coef_mat_emoji.index).apply(
                        lambda x: emoji.emojize(f":{x}:", use_aliases=True))

You would have to timeit with your actual data to see if this is any faster. It might be that the call to emoji.emojize will dominate anyway.