Questions tagged [vectorization]
Use this tag if your code contains vectorized operations - where functions operate on whole arrays in one go. If it doesn't, but you want it to, then you should make those changes (perhaps with help from elsewhere) before presenting code for review.
103 questions
2
votes
1
answer
190
views
Computing the angle between two vectors (vectorized) for small angles and with few copies
I am implementing a function that computes the angle between two vectors when given two n-dimensional arrays and an axis along which to operate. I want to do this with as few copies as possible, and ...
- 65.2k
3
votes
1
answer
103
views
Vectorizing a working custom similarity function further using numpy
I am new to python, and even more new to vectorization. I have attempted to vectorize a custom similarity function that should return a matrix of pairwise similarities between each row in an input ...
- 65.2k
5
votes
2
answers
207
views
Snake game from the viewpoint of the snake
I wrote a little game of Snake where you can see the field in which the snake moves fixed and you can also see the "viewpoint" of the snake, which is basically calculating the positions of ...
- 6,070
5
votes
1
answer
231
views
Implement vectorization instead of nested loops on dataframe
I have a dataset ('sample_data.csv') of the form below:
...
- 65.2k
3
votes
1
answer
204
views
Custom model intended to be used for curve fitting: vectorization vs for loop and JIT Numba capabilities
I am new to community and please pardon me if I didn't provide information as intended.
This code is supposed to be creating a custom model which will be used with ...
- 65.2k
3
votes
2
answers
111
views
C - SIMD Code to invert a transformation matrix
I am writing a maths library for a raytracer project, and so I'm trying to make my heavy operations (like matrix inverse) more optimised. After doing some research, I discovered this trick to invert a ...
4
votes
1
answer
188
views
Finding specific promotions from two columns [closed]
I'm trying to build a function that identifies those who are promoted into a list of jobcodes, or are promoted within that list of jobcodes.
Initially I was using ...
- 2,176
3
votes
1
answer
74
views
Calculating premium splits for policies
Looking for a better approach to write below transformation using Python. Is it possible to avoid loop and still achieve the desired output?
It is too slow for 10 million rows.
...
- 65.2k
2
votes
1
answer
89
views
Agent based Dengue transmission simulation
I'm trying to implement the dynamics of Dengue transmission between mosquitoes and humans. I decided create two classes, one for the human population and another for the mosquitoes. Both are based on ...
- 34.8k
4
votes
1
answer
182
views
Vectorized 16-bit addition in Standard C
The idea is to add a given 16-bit number N to each element of an array of 16-bit integers of arbitrary length, taking advantage of 64-bit integer types and instructions to perform the additions 4 at a ...
- 8,344
1
vote
2
answers
152
views
Replace personal names and addresses with company ones
The problem:
I am given a data frame. Somewhere in that dataframe there is 3*N
number of columns that I need to modify based on a condition. The
columns of interest look like this:
names_1
address_1
...
- 65.2k
4
votes
1
answer
710
views
Implementation of Iterative Closest Point in C++
Here, is my implementation of Iterative Closest Point algorithm in C++. The code is written using the Eigen library. I have tried to implement an efficient coding methodology best to my knowledge ...
- 24.8k
6
votes
1
answer
311
views
How to make my groupby and transpose operations efficient?
I have a DataFrame of size 3,745,802 rows and 30 columns. I would like to perform certain groupby and ...
- 34.8k
1
vote
1
answer
82
views
Make unique id based on text data column with similarity scoring
I have the following dataframe:
...
1
vote
1
answer
709
views
Exponentially-weighted moving mean and standard deviation of an irregularly-spaced weighted time series
The following numpy/python function computes exponentially-weighted moving mean and standard deviation of an irregularly-spaced weighted time series. I want to make it faster by getting rid of the ...
- 65.2k
0
votes
2
answers
60
views
Tips to Finetuning to increase the GFLOPS of a SIMD kernel
Given an Image I with dimensions as {rows, columns}, for every pixel in the image, the following needs to be computed -
...
- 61.7k
3
votes
1
answer
169
views
Matlab vs C: Tensorproduct or Vec-trick (multiple times)
I am searching for a more efficient way to calculate the so called vec-trick used in Tensor algebra, see Wikipedia.
Introduction:
Suppose you have a matrix vector multiplication, where a matrix C ...
- 6,237
11
votes
3
answers
2k
views
My blit function for my own graphics library
Here is a blit function for a graphics library I made.
I've built a small graphics library that uses an palette-indexed spritesheet to hold all of the game's sprites. The blit function copies parts of ...
- 113
1
vote
1
answer
58
views
Correcting coordinates of an image with polynomial root in python
I want to optimize this for loop for correcting coordinates of an image, it takes too long which is not suited for my system. I have done some profiling, the numpy roots is taking most of the time (...
- 28.3k
1
vote
0
answers
141
views
Faster Tridiagonal Matrix Multiplication
I have a function that I've been trying to make faster that computes \$XA + B\$, where \$A \in \mathbb{R}^{n \times n}\$ is a tridiagonal matrix, and \$X, B \in \mathbb{R}^{m \times n}\$. In the code, ...
- 211
4
votes
1
answer
55
views
optimize binary classification method for speed
I have the following code for determining TP, TN, FP and FN values for binary classification given two sparse vectors as input (using the sparse library):
...
- 41.1k
4
votes
1
answer
96
views
Can this numpy code be vectorized?
I've written the following function to produce n realizations of a CIR process for a given set of parameters:
...
- 2,331
7
votes
0
answers
441
views
C# Generic high performance vectorized math operations
I'm writing a library which simplifies the usage of intrinsics in C#.
It's a generic library which supports all numeric types.
The goal of this library is to perform these operations at the highest ...
4
votes
1
answer
153
views
Vectorized crosstabulation in Python for two arrays with two categories each
I have two Python lists, label and presence. I want to do cross-tabulation and get the count for each block out of four, such as ...
Mast♦
- 13.5k
2
votes
3
answers
3k
views
Scaling and incrementing non-zero elements of a NumPy matrix
I have a NumPy matrix C and want create a copy of it cPrime, which has some operation of the original matrix to all non-zero ...
- 121
10
votes
2
answers
410
views
N-Body Optimization
I've created a serial C++ code for gravitational N-Body calculation. Because I expect to have upwards of 8-71 sparse bodies (ie, where Barnes-Hut is not necessarily practical) and running for long ...
CommunityBot
- 1
2
votes
0
answers
43
views
Vectorize a function that, once per row, updates multiple rows in data.table
Requirements
I have a table of the different diagnoses a patient has received, with the date the diagnosis was first recorded. It so happens that in medicine, diagnoses are recorded as codes from a ...
- 201
1
vote
0
answers
56
views
Vectorization of Mersenne Twister? [closed]
Ahoy friends.
Right now i'm selling a huge amount of HDDs and usually i used DBAN or nwipe to perform this actions.
Unfortunately the PRNG Mersenne Twister is quite slow, so i get a throughput of ...
- 11
-1
votes
1
answer
516
views
Efficiently shift elements right by one in an __mm256 vector [closed]
The code below correctly shifts v256f_stats by one and puts the results into v256f_x3. This is then used to get the delta ...
- 10.7k
2
votes
1
answer
209
views
Vectorizing equations for fsolve
I have a system of 49 equations and 49 unknowns (\$x_{1},...,x_{49}\$). All equations have the following form.
\$x_{s}\sum_{r=1}^{R}\sum_{t=1}^{T}a_{s,t}b_{rs,t}=\sum_{r=1}^{R}\sum_{t=1}^{T}a_{r,t}...
- 241
1
vote
1
answer
93
views
vectorize 2D gradient with spatially varying bins
The following code takes in some values ssh and solves the equations for the geostrophic motion (slide 8 here).
The main part of the code is the computation of the ...
- 6,549
11
votes
1
answer
612
views
AVX Vectorized Multi-threaded Mandelbrot Renderer
I wrote a Mandelbrot renderer in C++ that uses AVX intrinsics to calculate 8 pixels per thread in parallel. It can compute 1080p frames with 256 max iterations in about 40 milliseconds on a 2 core 4 ...
- 1,725
2
votes
1
answer
164
views
Vectorization, 7-bit encoding
I was curious how vectorization works and how to write suck code. To start simple I choose 7-bit encoding as my test suspect. I'm not expecting improved throughput in any way. My implementation works ...
- 10.7k
1
vote
0
answers
205
views
condensed nearest centroid classifier in numpy
This is my attempt to write a numpy-optimized version of a nearest centroid classifier to classify some images from the MNIST data set of handwritten digits. I am ...
- 127
4
votes
2
answers
1k
views
PyTorch Vectorized Implementation for Thresholding and Computing Jaccard Index
I have been trying to optimize a code snippet which finds the optimal threshold value in a n_patch * 256 * 256 probability map to get the highest Jaccard index ...
- 73
6
votes
2
answers
3k
views
Remove outliers from a point cloud
This function accepts a cloud of points, and returns those points that are within delta distance of the average (mean) position.
...
- 103
4
votes
1
answer
1k
views
C# System.Numerics.Vector<T> Clamp
The software that I develop uses large floating-point arrays up to the maximum size that can be allocated in C#. I have a large number of algorithms such as convolutions and filters that get executed ...
CommunityBot
- 1
-2
votes
2
answers
2k
views
Converting Array of `Float32` (`float`) to Array of `UINT8` (`unsigned char`) Using AVX2 [closed]
Given input array of Float32 (float) with numElements how could one efficiently convert it ...
- 582
0
votes
1
answer
1k
views
Converting Array of `UINT8` (`unsigned char`) to Array of `Float32` (`float`) Using AVX2
Given input array of UINT8 (unsigned char) with numElements how could one efficiently ...
- 25.3k
7
votes
2
answers
4k
views
Rolling regressions in R
I want to run rolling regressions over each group and store the coefficient. The following works, but it's slow, since I have too many series and I want to run too many regressions for each group.
...
3
votes
0
answers
72
views
Optimising multiplication of matrix rows with matrix slice/page
I've recently vectorized most of the code for an optimization problem but this one still eludes me. As the procedure and not the actual values are important here, I have replaced the original matrices ...
- 131
1
vote
1
answer
499
views
Vectorized N-Dimensional Random Walk in Cartesian Coordinates
I have written a random-walk routine that I hope to build upon in the future. Before doing so, I was hoping to get some critique.
I believe the implementation is correct. I noticed that many other ...
CommunityBot
- 1
10
votes
1
answer
686
views
Vectorizing the product of an array of complex numbers
I am trying to write fast/optimal code to vectorize the product of an array of complex numbers. In simple C this would be:
...
- 144k
3
votes
1
answer
233
views
Vectorizing matrix operation instead of for loop in circular matrices
I have a vector \$X_k\$ and a matrix \$Y_{k,j}\$, where \$k \in \{1,\dots,K\}\$ and \$j \in \{1, \dots, J\}\$. They are circular, meaning that $$\eqalign{X_{k+K} &= X_k \\ Y_{k+K,j} &= Y_{k,j} ...
- 49.7k
4
votes
1
answer
334
views
Optimizations of C Code - Bilateral Filter
I'm working on a nice approximation for the Bilateral Filter.
I have a working code which runs pretty fast yet still I think much can be improved.
The code (C Code,...
CommunityBot
- 1
10
votes
2
answers
2k
views
Vectorized and Multi Threaded Image Convolution
I created code for Image Convolution. The code is in my Image Convolution GitHub Repository.
It includes the case for arbitrary Image Convolution and for Separable Kernel Convolution.
The code is a ...
- 10.7k
3
votes
1
answer
93
views
Generic pixel class to seamlessly alpha-blend and convert between different pixel structure layouts
Does what it says in the title. I just finished this and wanted to share with someone.
Looking for possible optimizations, bugs (most of it is tested to work) or any constructive criticism.
...
- 19.4k
4
votes
2
answers
611
views
Vectorizing a complex nested for loop in R (running models on different subsets of a data set, subsetting the data differently for each loop)
This is my first time posting so hopefully I manage to get my question across in a clear and concise manner. I'm not sure this is the right place but someone on StackOverflow suggested I ask here.
I ...
- 230
2
votes
1
answer
6k
views
Find first threshold crossing in numpy array
Given the following artificially generated data:
...
- 41.1k
3
votes
1
answer
69
views
Determining whether a list of pathways and its genes are all included in another list
I am evaluating if a list of pathways and its genes has pathways included in another pathways. So the way I thought about it is to create a function that does just a single comparison.
...
- 3,525