# Groupby count, then sum and get the percentage

I wrote this code.

It works, but I think there is a more elegant and Pythonic way to this task.

1. Groupby and count the different occurences
2. Get the sum of all the occurences
3. Divide each occurrence by the total of the occurrences and get the percentage

#Creating the dataframe
##The cluster column represent centroid labels of a clustering alghoritm
df=pd.DataFrame({'char':['a','b','c','d','e'], 'cluster':[1,1,2,2,2]})

#Counting the frequency of each labels
cluster_count=df.groupby('cluster').count()

#Calculating the sum of the frequency
cluster_sum=cluster_count.sum()

#Normalizing the frequency
cluster_prct=cluster_count.char.apply(lambda x: 100*x/cluster_sum)

print(cluster_prct)


Output:

cluster
1        40.0
2        60.0

• For what Python version did you write this? Where does the output come from?
– Mast
Jun 4, 2019 at 11:20
• @Mast for Python 3.0. The output comes out from print(cluster_prct). I edited the question. Jun 4, 2019 at 11:24

I think your code is already nearly optimal and Pythonic. But there is some little things to improve:

• cluster_count.sum() returns you a Series object so if you are working with it outside the Pandas, it is better to specify the column: cluster_count.char.sum(). This way you will get an ordinary Python integer.
• Pandas has an ability to manipulate with columns directly so instead of apply function usage you can just write arithmetical operations with column itself: cluster_count.char = cluster_count.char * 100 / cluster_sum (note that this line of code is in-place work).

Here is the final code:

df = pd.DataFrame({'char':['a','b','c','d','e'], 'cluster':[1,1,2,2,2]})
cluster_count=df.groupby('cluster').count()
cluster_sum=sum(cluster_count.char)
cluster_count.char = cluster_count.char * 100 / cluster_sum


Edit 1: You can do the magic even without cluster_sum variable, just in one line of code:

cluster_count.char = cluster_count.char * 100 / cluster_count.char.sum()

But I am not sure about its perfomance (it can probably recalculate the sum for each group).

Just to add in my 2 cents here:

You can approach this with series.value_counts() which has a normalize parameter.

From the docs:

normalize : boolean, default False If True then the object returned will contain the relative frequencies of the unique values.

Using this we can do:

s=df.cluster.value_counts(normalize=True,sort=False).mul(100) # mul(100) is == *100
s.index.name,s.name='cluster','percentage_' #setting the name of index and series
print(s.to_frame()) #series.to_frame() returns a dataframe


          percentage_
cluster
1               40.0
2               60.0