Groupby count, then sum and get the percentage

Question

I wrote this code.

It works, but I think there is a more elegant and Pythonic way to this task.

1. Groupby and count the different occurences
2. Get the sum of all the occurences

3. Divide each occurrence by the total of the occurrences and get the percentage

   #Creating the dataframe 
   ##The cluster column represent centroid labels of a clustering alghoritm 
   df=pd.DataFrame({'char':['a','b','c','d','e'], 'cluster':[1,1,2,2,2]})
   
   #Counting the frequency of each labels
   cluster_count=df.groupby('cluster').count()
   
   #Calculating the sum of the frequency
   cluster_sum=cluster_count.sum()
   
   #Normalizing the frequency
   cluster_prct=cluster_count.char.apply(lambda x: 100*x/cluster_sum)
   
   print(cluster_prct)
   

Output:

cluster      
1        40.0
2        60.0

Answer 1

I think your code is already nearly optimal and Pythonic. But there is some little things to improve:

• cluster_count.sum() returns you a Series object so if you are working with it outside the Pandas, it is better to specify the column: cluster_count.char.sum(). This way you will get an ordinary Python integer.
• Pandas has an ability to manipulate with columns directly so instead of apply function usage you can just write arithmetical operations with column itself: cluster_count.char = cluster_count.char * 100 / cluster_sum (note that this line of code is in-place work).

Here is the final code:

df = pd.DataFrame({'char':['a','b','c','d','e'], 'cluster':[1,1,2,2,2]})
cluster_count=df.groupby('cluster').count()
cluster_sum=sum(cluster_count.char)
cluster_count.char = cluster_count.char * 100 / cluster_sum

---

Edit 1: You can do the magic even without cluster_sum variable, just in one line of code:

cluster_count.char = cluster_count.char * 100 / cluster_count.char.sum()

But I am not sure about its perfomance (it can probably recalculate the sum for each group).

Answer 2

Just to add in my 2 cents here:

You can approach this with series.value_counts() which has a normalize parameter.

From the docs:

normalize : boolean, default False If True then the object returned will contain the relative frequencies of the unique values.

Using this we can do:

s=df.cluster.value_counts(normalize=True,sort=False).mul(100) # mul(100) is == *100
s.index.name,s.name='cluster','percentage_' #setting the name of index and series
print(s.to_frame()) #series.to_frame() returns a dataframe

---

          percentage_
cluster             
1               40.0
2               60.0