40
It will certainly be faster if you vectorize the distance calculations:
def closest_node(node, nodes):
nodes = np.asarray(nodes)
dist_2 = np.sum((nodes - node)**2, axis=1)
return np.argmin(dist_2)
There may be some speed to gain, and a lot of clarity to lose, by using one of the dot product functions:
def closest_node(node, nodes):
nodes = ...
32
The NumPy Reference should be the first place you look when you have a problem like this. The operations you need are nearly always in there somewhere. And functions that you find while browsing the reference are sure to come in useful later in your NumPy career.
For the first row, you could use numpy.arange and numpy.repeat:
>>> np.repeat(np....
27
1. Introduction
This question is difficult because:
It's not clear what the function countlower does. It's always a good idea to write a docstring for a function, specifying what it does, what arguments it takes, and what it returns. (And test cases are always appreciated.)
It's not clear what the role of the arguments n1 and n2 is. The code in the post ...
27
NumPy provides numpy.interp for 1-dimensional linear interpolation. In this case, where you want to map the minimum element of the array to −1 and the maximum to +1, and other elements linearly in-between, you can write:
np.interp(a, (a.min(), a.max()), (-1, +1))
For more advanced kinds of interpolation, there's scipy.interpolate.
25
Vectorization with NumPy
When read with cv2.imread or skimage.io.imread or scipy.misc.imread, you would already have the image data as a NumPy array. Now, NumPy supports various vectorization capabilities, which we can use to speed up things quite a bit.
I. Crop to remove all black rows and columns across entire image
To solve our case, one method would be ...
25
A look at the second plot in the post shows that something has gone wrong. There are four points in a line here:
This shouldn't be possible, since "points that create a straight line when plotted" have been removed. How did this happen? A look at the \$x\$ axis shows the cause. The first plot has the \$x\$ coordinate running from 0 to 96, but the second ...
21
As you've discovered, looping over individual pixels in Python is very slow. You need to organize your computation so that it uses a series of NumPy (or SciPy, or Scikit-Image, or OpenCV) operations on the whole image.
In this case, you could use numpy.argwhere to find the bounding box of the non-black regions:
# Mask of non-black pixels (assuming image ...
19
All your code could be rewritten as:
from numpy import random
from scipy.spatial import distance
def closest_node(node, nodes):
closest_index = distance.cdist([node], nodes).argmin()
return nodes[closest_index]
a = random.randint(1000, size=(50000, 2))
some_pt = (1, 2)
closest_node(some_pt, a)
You can just write randint(1000) instead of randint(...
19
Your code does not run: there are missing import statements:
import numpy as np
import scipy.spatial.distance
Your algorithms compute different results, so some of them must be wrong!
>>> vector = np.array([1, 2, 3, 4, 5])
>>> matrix = np.array([[7, 5, 8, 1, 9], [6, 6, 4, 0, 8]])
>>> cos_matrix_multiplication(matrix, vector)
...
18
On my computer it takes 1.745 seconds to run the code in the post.
There's no need for the array of random indexes to be two-dimensional:
IDX = np.random.randint(0,P,(N,1))
In fact this is harmful for performance, because it means that x0[i] is an array of length 1 (not a scalar) and so img[x0[i]+x,y0[i]+y] requires "fancy indexing" which is slower than ...
17
1. Review
No docstring. What does this function do? What parameters does it take? What shape must the depth_image parameter be? What does it return?
In Python, there is no need for a semi-colon at the end of a statement (unless another statement follows on the same line) and it is best to omit it.
It seems that depth_image is required to have three ...
16
1. Review
It's not recommended to use using namespace std; — the problem is that this imports all of the identifiers from std, and some of these may shadow names from other modules that you need to use. See this question on Stack Overflow.
The code doesn't check for success/failure of many of the functions it calls. These can all fail:
new long double[SIZE]...
15
1. Analysis
In the general case, the problem of finding the sample of words with the "most even" distribution of letters is NP-hard. Here I'm considering a general instance of this problem to be:
Given an alphabet \$ Σ \$, a set \$ W \$ of 4-letter words over that alphabet, and an integer \$ n ≤ \left|W\right| \$, find the subset \$ W^* ⊆ W \$ with size \...
15
Your original code can be rewritten as:
mask = np.all(img == (255, 127, 63), axis=-1)
It is a little cleaner, but not more efficient, as it still has to allocate a mask of the same size as the image.
14
In the heat function, simply vectorizing the inner loop, drops the time from 340 sec to 56 sec, a 6x improvement. It starts by defining the first column of Z, and calculates the next column from that (modeling heat diffusion).
def heat(D,u0,q,tdim):
xdim = np.size(u0)
Z = np.zeros([xdim,tdim])
Z[:,0]=u0;
for i in range(1,tdim):
#...
14
You can decrease Python's overhead by parsing the args tuple and creating the return value manually.
Try changing these portions of your code:
/* Parse the input tuple */
if (!PyArg_ParseTuple(args, "OO", &x_obj, &y_obj))
return NULL;
<< code snipped >>
/* Build the output tuple */
PyObject *ret = Py_BuildValue("d", value);
return ...
14
1. Code review
There's no documentation. What do your functions do and how am I supposed to call them?
There are no test cases. What gives you confidence that the algorithm is correct?
The algorithm is called "\$k\$-means" (not "\$k\$-mean") so I would name the functions accordingly.
The do_print mechanism looks as if it was added to help with debugging and ...
13
1. Introduction
Thanks for running the profiler. As you can see from the output, most of the runtime is being spent in your containing_tet function.
The first thing to say is that you have made this question unnecessarily difficult for us because your functions have no documentation. We have to read and reverse-engineer your code to try to figure out what ...
13
I'll just be cheeky and post a slightly modified version of my SO answer here.
So first things first, you want to get rid of the loops. They are slow to execute.
The first loop:
for x in range(rows):
for y in range(cols):
if Z[x][y] == 1:
if (N[x][y] < 2) or (N[x][y] > 3):
Z[x][y] = 0
...
13
Just create a list of 18 ones and 18 twos, shuffle it, then reshape to 6x6:
from random import shuffle
from numpy import reshape
nums = [1]*18 + [2]*18
shuffle(nums)
arr = reshape(nums, (6, 6))
Produces (for example):
array([[1, 2, 2, 1, 2, 1],
[2, 2, 2, 2, 2, 1],
[2, 2, 2, 1, 1, 2],
[1, 1, 1, 1, 2, 2],
[2, 1, 1, 2, 1, 1],
...
13
The key thing to remember when working with numerical code is that the CPython interpreter is pretty slow (it trades speed for flexibility) and so you must avoid running in the interpreter whenever possible. Instead of iterating in slow Python bytecode using for x in ..., operate on whole arrays by calling the appropriate NumPy function or method, which ...
13
Use conditional indexing:
RED, GREEN, BLUE = (2, 1, 0) # Your numbers
empty_img = numpy.zeros((height, width, 3), numpy.uint8)
reds = img[:, :, RED]
greens = img[:, :, GREEN]
blues = img[:, :, BLUE]
empty_img[(greens < 35) | (reds <= greens >= blues)][BLUE] = 255
Edit:
empty_img[(greens < 35) | ((reds <= greens) & (blues <= greens)...
12
The function scipy.ndimage.find_objects returns exactly the slices that you are looking for:
>>> a = np.array([1, 1, 1, 2, 2, 3])
>>> scipy.ndimage.find_objects(a)
[(slice(0, 3, None),), (slice(3, 5, None),), (slice(5, 6, None),)]
It's hard to make a fair timing comparison without knowing exactly how you are going to use these slices, but ...
12
There's a simpler way to create the empty image using numpy.zeros_like:
empty_img = numpy.zeros_like(img)
As Austin Hastings correctly pointed out, the trick is to use vectorized operations provided by numpy:
RED, GREEN, BLUE = (2, 1, 0)
reds = img[:, :, RED]
greens = img[:, :, GREEN]
blues = img[:, :, BLUE]
mask = (greens < 35) | (reds > greens) |...
12
Loops over large arrays are not really a good idea in Python. This is why your original list comprehension is not terribly fast.
Your numpy version is loop free, but as far as I know, np.repeat actually makes copies of your data, which again, is really inefficient. An alternative would be to use np.tile, which maybe does not need to copy the data. But we ...
12
Here is a review of the solution.
^ is xor in Python. It is not for computation of exponentials.
When running code outside a method / class, it is a good practice to put the code inside a main guard. See here for more explanation.
if __name__ == "__main__":
...
When you are providing quick answers on a forum, the guard might not always be ...
11
This is really a graph problem! Draw a graph with a vertex for every number in your list, and an edge between vertices \$a\$ and \$b\$ if there is a pair \$(a, b)\$ or \$(b, a)\$ in your list:
What you want to do is to find a path in this graph that visits all the edges. Such a path is known as an Eulerian path. Your condition that each number appears ...
11
Everything apart from the for loop in your code is already vectorized which leaves only the for loop to be optimized.
You define a counter which is raised by 1 every time the entry of clock_ticks_coarse is 1. There exists a function cumsum in numpy which sums all entries of the argument like this:
import numpy as np
x = np.array([1,3,0,5,7])
np.cumsum(x)
...
11
matplotlib adding stuff to the current figure is because you are not using the OO-interface. It is slightly clunkier, but allows way more freedom
In my view, you are mixing several things up. I would seperate the generation of the timings, the aggregation of these results and the plotting. In this way, if you want to change to another plotting library (...
11
You’re using the wrong tool for the job. Basically, you do all the computation in Python, use numpy for intermediate storage and pandas for display.
Instead, you should compute the list of tribonacci numbers and from there on use pandas for anything else as it would be much more efficient / readable. I’d keep building the tribonacci numbers in Python as I ...
answered Jan 29 '18 at 16:46
301_Moved_Permanently
27.5k3 gold badges39 silver badges94 bronze badges
Only top voted, non community-wiki answers of a minimum length are eligible
