I need to randomly erase 33% of the non-zero elements in each line of a matrix.
Side note: The end goal is to train a denoising autoencoder to remove this noise.
import numpy as np matrix = np.random.rand(1000,3000) for i in range(matrix.shape): clean = matrix[i, :] # original matrix line # find non zero elements msk = np.nonzero(clean) assert sum(msk) != 0 # keep 66% of them idx = np.random.randint(0, len(msk), size=max(1, len(msk)//3)) #erase at least 1 msk = np.delete(msk, idx) dirty = clean dirty = [j if i in msk else 0 for i,j in enumerate(dirty)] assert sum(clean-dirty) != 0 #save clean and dirty #...
My guess is that the bottleneck is drawing random numbers at each iteration.
Is there an alternative way to do this?