0
\$\begingroup\$

I was just wondering if there is a way to speed up the performances of this for loops in Python.

I'm trying to process an image to get the color-moments without using libraries. It takes about 12sec to do the calculate_mean and calculate_standard_deviation functions for each part of the image.

import math

import cv2
import numpy as np
parts = 2
new_height = int(img.shape[0]/parts)
new_width = int(img.shape[1]/parts)

for i in range (0,img.shape[0],new_height):
    for j in range(0,img.shape[1],new_width):
        color_moments = [0,0,0,0,0,0,0,0,0]
        cropped_image = img[i:i+new_height,j:j+new_width]
        yuv_image = cv2.cvtColor(cropped_image,cv2.COLOR_BGR2YUV)
        Y,U,V = cv2.split(yuv_image)

        pixel_image_y = np.array(Y).flatten()
        pixel_image_u = np.array(U).flatten()
        pixel_image_v = np.array(V).flatten()

        calculate_mean(pixel_image_y,pixel_image_u,pixel_image_v,color_moments)
        calculate_standard_deviation(pixel_image_y, pixel_image_u, pixel_image_v, color_moments) 

And this are the two functions:

def calculate_mean(pixel_image_y,pixel_image_u,pixel_image_v,color_moments):
    for p in pixel_image_y:
        color_moments[0]+=(1/(new_height*new_width))*int(p)
    for p in pixel_image_u:
        color_moments[1]+=(1/(new_height*new_width))*int(p)
    for p in pixel_image_v:
        color_moments[2]+=(1/(new_height*new_width))*int(p)

def calculate_standard_deviation(pixel_image_y,pixel_image_u,pixel_image_v,color_moments):
    temp = [0,0,0]
    for p in pixel_image_y:
        temp[0]+=(p-color_moments[0])**2
    color_moments[3] = math.sqrt((1/(new_height*new_width))*temp[0])
    for p in pixel_image_u:
        temp[1]+=(p-color_moments[1])**2
    color_moments[4] = math.sqrt((1/(new_height*new_width))*temp[1])
    for p in pixel_image_v:
        temp[2]+=(p-color_moments[2])**2
    color_moments[5] = math.sqrt((1/(new_height*new_width))*temp[2])

\$\endgroup\$
6
  • 1
    \$\begingroup\$ is color_moments reset for each iteration, because that is not clear from the code here. Why not just use np.mean and np.std \$\endgroup\$ – Maarten Fabré May 27 '20 at 10:21
  • \$\begingroup\$ Yes, color_moments is reset for each iteration. I'm trying to not use the libraries \$\endgroup\$ – Enrico Mosca May 27 '20 at 10:26
  • 2
    \$\begingroup\$ Bolding "without using libraries" seems really odd. Like really really really odd, since you're using numpy and opencv. Please explain why you don't want to use these libraries and which libraries you can use. Because currently your description says one thing and your code says another. \$\endgroup\$ – Peilonrayz May 27 '20 at 10:27
  • \$\begingroup\$ I need to calculate the mean and standard deviation without the libraries given by numpy. I edited the code to show how I reset the color_moments \$\endgroup\$ – Enrico Mosca May 27 '20 at 13:01
  • 1
    \$\begingroup\$ Please add your imports and all other relevant code. \$\endgroup\$ – Mast May 27 '20 at 17:54
0
\$\begingroup\$
  1. Python 2 is end of life, many core libraries and tools are dropping Python 2 support.
  2. Python is dumb. If you tell it to calculate (1/(new_height*new_width) each and every loop, then Python will.
  3. Your style is not Pythonic, you should have more whitespace.
  4. If we change one of the for loops in calculate_mean to math, we can see we can that ¶2 isn't needed inside the loop.

    for p in pixel_image_y:
        color_moments[0]+=(1/(new_height*new_width))*int(p)
    

    $$ \Sigma \frac{\lfloor p \rfloor}{hw} = \frac{\Sigma \lfloor p \rfloor}{hw} $$

  5. We can utilize numpy to perform the loop, in calculate_standard_deviation, in C.

    color_moments[3] = (
        np.sum((pixel_image_y - color_moments[0]) ** 2)
        * math.sqrt((1/(new_height*new_width))
    )
    
  6. You should change calculate_mean and calculate_standard_deviation to either work in one dimension or all dimensions.
  7. You should remove color_moments from your code. You should return if you need to return.
  8. You can determine new_height*new_width by the values you enter in your function. No need to pass them.
def mean(values):
    return sum(int(v) for v in values) / len(values)

def std_deviation(values):
    return np.sum((values - mean(values)) ** 2) / math.sqrt(len(values))
\$\endgroup\$
3
  • \$\begingroup\$ I don't understand the fourth point \$\endgroup\$ – Enrico Mosca May 27 '20 at 17:36
  • \$\begingroup\$ @EnricoMosca If we have the list [1, 0, 1, 0, 1] then this can be seen as \$\Sigma \frac{\lfloor p \rfloor}{5} = \frac{1}{5} + \frac{0}{5} + \frac{1}{5} + \frac{0}{5} + \frac{1}{5}\$, we can simplify this to move the fraction away from the sum \$\frac{\Sigma \lfloor p \rfloor}{5} = \frac{1 + 0 + 1 + 0 + 1}{5}\$. \$\endgroup\$ – Peilonrayz May 27 '20 at 17:42
  • \$\begingroup\$ Thank you so much for your precious help!! \$\endgroup\$ – Enrico Mosca May 28 '20 at 8:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.