4
\$\begingroup\$

I wrote this code.

It works, but I think there is a more elegant and Pythonic way to this task.

  1. Groupby and count the different occurences
  2. Get the sum of all the occurences
  3. Divide each occurrence by the total of the occurrences and get the percentage

    #Creating the dataframe 
    ##The cluster column represent centroid labels of a clustering alghoritm 
    df=pd.DataFrame({'char':['a','b','c','d','e'], 'cluster':[1,1,2,2,2]})
    
    #Counting the frequency of each labels
    cluster_count=df.groupby('cluster').count()
    
    #Calculating the sum of the frequency
    cluster_sum=cluster_count.sum()
    
    #Normalizing the frequency
    cluster_prct=cluster_count.char.apply(lambda x: 100*x/cluster_sum)
    
    print(cluster_prct)
    

Output:

cluster      
1        40.0
2        60.0
\$\endgroup\$
  • \$\begingroup\$ For what Python version did you write this? Where does the output come from? \$\endgroup\$ – Mast Jun 4 at 11:20
  • \$\begingroup\$ @Mast for Python 3.0. The output comes out from print(cluster_prct). I edited the question. \$\endgroup\$ – Andrea Ciufo Jun 4 at 11:24
4
\$\begingroup\$

I think your code is already nearly optimal and Pythonic. But there is some little things to improve:

  • cluster_count.sum() returns you a Series object so if you are working with it outside the Pandas, it is better to specify the column: cluster_count.char.sum(). This way you will get an ordinary Python integer.
  • Pandas has an ability to manipulate with columns directly so instead of apply function usage you can just write arithmetical operations with column itself: cluster_count.char = cluster_count.char * 100 / cluster_sum (note that this line of code is in-place work).

Here is the final code:

df = pd.DataFrame({'char':['a','b','c','d','e'], 'cluster':[1,1,2,2,2]})
cluster_count=df.groupby('cluster').count()
cluster_sum=sum(cluster_count.char)
cluster_count.char = cluster_count.char * 100 / cluster_sum

Edit 1: You can do the magic even without cluster_sum variable, just in one line of code:

cluster_count.char = cluster_count.char * 100 / cluster_count.char.sum()

But I am not sure about its perfomance (it can probably recalculate the sum for each group).

\$\endgroup\$
1
\$\begingroup\$

Just to add in my 2 cents here:

You can approach this with series.value_counts() which has a normalize parameter.

From the docs:

normalize : boolean, default False If True then the object returned will contain the relative frequencies of the unique values.

Using this we can do:

s=df.cluster.value_counts(normalize=True,sort=False).mul(100) # mul(100) is == *100
s.index.name,s.name='cluster','percentage_' #setting the name of index and series
print(s.to_frame()) #series.to_frame() returns a dataframe

          percentage_
cluster             
1               40.0
2               60.0
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.