Skip to main content
added 314 characters in body
Source Link
hpaulj
  • 1.5k
  • 1
  • 9
  • 16

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.

Why is the difference time difference between a view and a copy significant? If you do it a few times in the program, it shouldn't matter. If you do this deletion repeatedly, I'd question the larger program design. Could, you for example, accumulate the deletion indices, and perform the deletion step just once?


A few timings:

In [17]: arr=np.arange(1000)

In [18]: timeit arr[np.r_[:500,501:1000]].shape
10000 loops, best of 3: 55.7 us per loop

In [19]: timeit arr.take(np.r_[:500,501:1000]).shape
10000 loops, best of 3: 44.2 us per loop

In [20]: timeit np.r_[:500,501:1000]
10000 loops, best of 3: 36.3 us per loop 

In [23]: timeit ind=np.ones(arr.shape[0],bool);ind[500]=False;arr[ind].shape
100000 loops, best of 3: 12.8 us per loop

Oops, the boolean index is faster in this test case.

Best yet:

In [26]: timeit np.concatenate((arr[:500],arr[501:])).shape
100000 loops, best of 3: 4.61 us per loop

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.

Why is the difference time difference between a view and a copy significant? If you do it a few times in the program, it shouldn't matter. If you do this deletion repeatedly, I'd question the larger program design. Could, you for example, accumulate the deletion indices, and perform the deletion step just once?


A few timings:

In [17]: arr=np.arange(1000)

In [18]: timeit arr[np.r_[:500,501:1000]].shape
10000 loops, best of 3: 55.7 us per loop

In [19]: timeit arr.take(np.r_[:500,501:1000]).shape
10000 loops, best of 3: 44.2 us per loop

In [20]: timeit np.r_[:500,501:1000]
10000 loops, best of 3: 36.3 us per loop

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.

Why is the difference time difference between a view and a copy significant? If you do it a few times in the program, it shouldn't matter. If you do this deletion repeatedly, I'd question the larger program design. Could, you for example, accumulate the deletion indices, and perform the deletion step just once?


A few timings:

In [17]: arr=np.arange(1000)

In [18]: timeit arr[np.r_[:500,501:1000]].shape
10000 loops, best of 3: 55.7 us per loop

In [19]: timeit arr.take(np.r_[:500,501:1000]).shape
10000 loops, best of 3: 44.2 us per loop

In [20]: timeit np.r_[:500,501:1000]
10000 loops, best of 3: 36.3 us per loop 

In [23]: timeit ind=np.ones(arr.shape[0],bool);ind[500]=False;arr[ind].shape
100000 loops, best of 3: 12.8 us per loop

Oops, the boolean index is faster in this test case.

Best yet:

In [26]: timeit np.concatenate((arr[:500],arr[501:])).shape
100000 loops, best of 3: 4.61 us per loop
added 317 characters in body
Source Link
hpaulj
  • 1.5k
  • 1
  • 9
  • 16

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.

Why is the difference time difference between a view and a copy significant? If you do it a few times in the program, it shouldn't matter. If you do this deletion repeatedly, I'd question the larger program design. Could, you for example, accumulate the deletion indices, and perform the deletion step just once?


A few timings:

In [17]: arr=np.arange(1000)

In [18]: timeit arr[np.r_[:500,501:1000]].shape
10000 loops, best of 3: 55.7 us per loop

In [19]: timeit arr.take(np.r_[:500,501:1000]).shape
10000 loops, best of 3: 44.2 us per loop

In [20]: timeit np.r_[:500,501:1000]
10000 loops, best of 3: 36.3 us per loop

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.

Why is the difference time difference between a view and a copy significant? If you do it a few times in the program, it shouldn't matter. If you do this deletion repeatedly, I'd question the larger program design. Could, you for example, accumulate the deletion indices, and perform the deletion step just once?


A few timings:

In [17]: arr=np.arange(1000)

In [18]: timeit arr[np.r_[:500,501:1000]].shape
10000 loops, best of 3: 55.7 us per loop

In [19]: timeit arr.take(np.r_[:500,501:1000]).shape
10000 loops, best of 3: 44.2 us per loop

In [20]: timeit np.r_[:500,501:1000]
10000 loops, best of 3: 36.3 us per loop
Source Link
hpaulj
  • 1.5k
  • 1
  • 9
  • 16

Since the result's buffer will have a gap, compared to the original, it will have to be a copy. I believe delete takes different approaches depending on the inputs.

One approach is a boolean index, e.g.

ind = np.ones((10,), bool)
ind[n] = False
A1 = A[ind,:]

Another is to do the equivalent with index values

ind = range(n) + range(n+1:A.shape[0]] # using list concatenate
A1 = A[ind,:]

And as you note, using that index with take may be faster than direct indexing. I doubt if the difference is big, but I haven't timed it recently.

ind could also made by concatenation of 1d arrays. Alternatively, index the two parts, and concatenate them:

 np.concatenate([A[:n,:],A[n+1:],axis=0)

The inputs to concatenate are slices, but result is a copy.

 np.r_[0:n, n+1:A.shape[0]]

is a convenient way of generating the integer index list - but not necessarily a speed solution.