1
\$\begingroup\$

I was working with twitter data, and extracted every emoji from the data. But, when I passed that data through CountVectorizer, colons where substracted from the strings. So, the string emoji :ok_hand: :thumbs_up: turned into ok_hand thumbs_up. I wanted to re-add those colons so then I could emojize them back. I managed to do that, but I'm quite sure my method is very inefficient. The emojis are the indexes of a coefficients DataFrame, like this:

    index              coef

ok_hand thumbs_up      0.4
    airplane           0.2

So what I did was this:

to_emojize=pd.Series(coef_mat_emoji.index)
to_emojize=to_emojize.apply(lambda x: x.split())
to_emojize=to_emojize.apply(lambda x:[':'+i+':' for i in x])
to_emojize=to_emojize.apply(lambda x: emoji.emojize(x, use_aliases=True))
coef_mat_emoji.index=to_emojize

Is there a better way to do this?

\$\endgroup\$
  • \$\begingroup\$ I think you could put all three steps into a single function and apply them in one iteration over the data. \$\endgroup\$ – AlexV Apr 16 at 19:56
3
\$\begingroup\$

Both pandas.Series and pandas.Index have vectorized string additions. You can just do:

to_emojize = pd.Series(":" + coef_mat_emoji.index + ":")
coef_mat_emoji.index = to_emojize.apply(emoji.emojize, use_aliases=True)

Note that pandas.Series.apply passes any additional keyword arguments along to the function, so there is no need for the lambda here at all.

This will create an intermediate series from the first addition, which might not be the most memory efficient way to do this. But it is the easiest and most readable, so unless you run out of memory with this, this is what I would use.

Alternatively you could put it all into one apply call (Python 3.6+ for f-strings):

coef_mat_emoji.index = pd.Series(coef_mat_emoji.index).apply(
                        lambda x: emoji.emojize(f":{x}:", use_aliases=True))

You would have to timeit with your actual data to see if this is any faster. It might be that the call to emoji.emojize will dominate anyway.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.