Questions tagged [numpy]
NumPy is the fundamental package for scientific computing with the programming language Python.
606
questions
-1
votes
0answers
33 views
Alternative way of data sorting to reduce excetion time in python [closed]
I have a large text file(~600 gb). File consist of float and/or int only.At first I create some empty array as a variable,
...
1
vote
1answer
70 views
2
votes
1answer
36 views
Construct a numpy array by repeating a 1 dimensional array sliced at different indices
I have a pandas dataframe (called base_mortality) with 1 column and n rows, which is of the following form:
...
1
vote
1answer
83 views
Getting the square of the difference between many points in a NumPy array
I have an array with 40000 numbers that are floats or ints. I need to perform some calculation.
To do this I have used nested for loop, but the code is really slow. Can I use something instead of ...
6
votes
3answers
145 views
Find the maximum price change over any 1 to 5 day rolling window
The code below solves the following task
Find the maximum price change over any 1 to 5 day rolling window, over a 1000 day period.
To be clear "any 1 to 5 day rolling window" means ...
2
votes
0answers
15 views
Getting Graph and Block Animation to Run Smoothly Together
I have been working on some code that produces a live graphic of a graph and blocks moving. It works in theory but does not look like how I want. The animation runs very slow when showing both the ...
2
votes
1answer
148 views
Smart Tic Tac Toe, a reinforcement learning approach
I'm currently familiarizing myself with reinforcement learning (RL). For convenience, instead of manually entering coordinates in the terminal, I created a very simple UI for testing trained agents ...
6
votes
0answers
53 views
Using get_dummies to create a Simple Recommender System - Cold Start
Question: was using get_dummies a good choice for converting categorical strings?
I used get_dummies to convert categorical ...
3
votes
1answer
83 views
Generating unique 2D numpy arrays with 2 adjacent row elements swapped while using BFS in Python
So I have a 12 x 6 2D numpy array for input which consists of 7 possible board objects (characters from 'abcdefg'). From a 2D numpy array I wish to generate all the possible unique 2D arrays, in whose ...
2
votes
1answer
27 views
Get the last activity per user per day in a dataframe
I have many users. Each time a user uses their smartphone it register them. I am determining the last time each user used their smartphone each day.
Additionally smartphone usages from 18:00 to 06:00, ...
4
votes
2answers
49 views
Analyzing the darkest pixels of an image in Python
This script takes an image copied from the clipboard and analyzes the n darkest pixels of the image. It will loop through each found value, prints out the value information and the quantity, then ...
1
vote
0answers
32 views
Calculation Correlation Time
I want to calculate the correlation time. First I calculate the auto-correlation function:
$$
\begin{align}
\chi(t) = &\frac{1}{t_{max}-t}\sum\limits_{t'=0}^{t_{max}-t}m(t')m(t'+t)\\
&-\frac{...
3
votes
1answer
39 views
Numpy array slicing/reshape/concatination
I'm sure my question exists on the internet, i just don't know the right formulations.
I have a data-sample as input for a NN. This sample has the shape (1, 8, 28, 80).
Basically it is 80 timesteps ...
0
votes
1answer
62 views
Image color-moment extractor [closed]
I was just wondering if there is a way to speed up the performances of this for loops in Python.
I'm trying to process an image to get the color-moments without using libraries.
It takes about 12sec ...
3
votes
2answers
58 views
Implementing a Steepest Descent Algorithm
I am teaching myself some coding, and as my first "big" project I tried implementing a Steepest Descent algorithm to minimize the Rosenbrock function:
$$f(x, y) = 100 (y - x^2)^2 + (1 - x)^2$$
The ...
4
votes
1answer
157 views
Jupyter notebook style help + code suggestions for pandas
I wanted to open source some code to scrape and analyze publicly-filed stock buys and sells from U.S. senators. I'm not familiar with code style for Jupyter notebooks or pandas in general. Would it be ...
5
votes
3answers
241 views
Efficient numpy array manipulation to convert an identity matrix to a permutation matrix
Task
I want to be able to generate the permutation matrix that splits a 1D array of consecutive numbers (i.e. even, odd, even, odd, even, odd, ...) into a 1D array where the first half are the evens, ...
5
votes
1answer
34 views
Applying gaussian blur on RGBA images
I designed the code and it works quite well for images of lower res. However my program takes a lot of time and ram to display higher res images (occupies 2GB RAM for 4k images and takes 20 minutes). ...
6
votes
2answers
110 views
Largest product in a grid - Project Euler 11
A stab at Project Euler Problem 11: Largest product in a grid
...
4
votes
1answer
66 views
Simulated annealing for magic square-of-squares puzzle
I'm working on a function that takes in a NumPy array containing only mutually distinct positive square numbers. I want the function to pick an element at random, find if there is a lower positive ...
2
votes
0answers
24 views
Profiling for Bézier curve calculations
Recently I posted an answer on a question about Bézier curve calculations. As a micro-synopsis: there are three implementations of De Casteljau's algorithm here, including the original poster's, AJ ...
1
vote
0answers
19 views
How to optimize code to take advantage of NumPy broadcasting when calculating pairwise correlation?
I'm trying to take advantage of NumPy broadcasting and backend array computations to significantly speed up this function. Unfortunately, it doesn't scale so well so I'm hoping to greatly improve the ...
2
votes
1answer
58 views
Fast way to calculate Standing Wave modes in NumPy
I am trying to plot the standing wave modes in a 2D box over time. The equation for this is:
$$ A(x,y,t) = \sum\limits_{n,m} sin \left( \frac{n \pi x}{L_x} \right) sin \left( \frac{m \pi y}{L_y} \...
1
vote
1answer
29 views
What would be the computationally faster way to implement this 2D numerical integration?
I am interested in doing a 2D numerical integration. Right now I am using the scipy.integrate.dblquad but it is very slow. Please see the code below. My need is to ...
4
votes
1answer
71 views
Code for wireless communication work
I need some pointers on how I can speed up my code, as of now it is incredible slow for larger inputs. The way the code works is that the file Loc_Circle_50U.txt contains the true locations of 50 ...
6
votes
3answers
500 views
Improvement on data normalization
I have a part of code that is loading a dataset and normalizing property values to [0, 1]. My implementation is:
...
3
votes
1answer
106 views
Building optimal portfolios for all combinations of stocks
Hi I recently wrote some code in python that does the following:
1.) Pulls stock closing data from yahoo finance for x number of stock
2.) finds all possible combinations of x stocks in groups of y ...
4
votes
1answer
105 views
Can this sampling code be vectorized or optimized?
I have some code that is slow because it loops over every row in an input matrix Y. Obviously, this code does not scale with the number of rows. I don't think it's ...
5
votes
1answer
56 views
Binning whilst avoiding the need for for-loops in Python?
I am working on a relatively simple binning program, where I take a 5D array and bin it based on two 3D arrays to create a contour plot. See the sample code below. In actuality, my arrays are of size <...
4
votes
1answer
78 views
How to stop a Python script if the input values do not make sense within the environment in which they will be analyzed?
I am not a developer, but I needed to write a script.
So I wrote it. Since I can't find an accessible tool that fulfills the same purpose, which I think may be helpful to others, I decided to ...
3
votes
0answers
63 views
Clean and fast MiniMax for numpy implementation of Tic-Tac-Toe
Tic-tac-toe is small enough to be solved completely but since I have a slow computer and this is an interesting exercise, I want to build the fastest possible minimax tic-tac-toe implementation in ...
2
votes
0answers
35 views
Convoluting 3D image with 2D
I have a single image of shape img.shape = (500, 439, 3)
The convolution function is
...
2
votes
1answer
176 views
AI In 2048 Game, Calculations so slow
Inspired by this answer on stackoverflow for AI implementation in 2048 game.
I have been trying to make AI calculate best move by playing the game given specific times in memory preceding by one ...
-1
votes
1answer
43 views
Trying to get every 6th set of 4 bytes from a python array [closed]
I have an array of bytes 6144 bytes long. I want to extract a set of 4 bytes every 6th set of 4 bytes . So every 24 bytes I want to extract 4 of those bytes and join them in a new array. So
...
2
votes
1answer
56 views
Forecasting stock market data using Support Vector Regression
I coded this Support Vector Regression (SVR) myself following some equations in a journal (see here, or here (not in English)). The loss function used by the journal and the code below is mean ...
0
votes
0answers
22 views
Sequential learning function running slowly
I'm new to Python and made this Sequential Learning function but it runs slowly. Do you know why this runs slowly and how to improve it?
...
5
votes
2answers
131 views
Loop through group and difference lags
I have time-series data (t) that I want to loop through each t and take the difference of the previous five lags ...
2
votes
1answer
111 views
Asteroids using pygame
I made the looks of asteroids using pygame. I still need to implement collisions but since it is running incredibly slow with 'amount' anything above 10. Please rip ...
1
vote
0answers
35 views
Python Implementation of Canopy Structural Dynamic Model using scipy.optimize.least_squares
I am using the least_squares() function from the scipy.optimize module to calibrate a Canopy structural dynamic model (CSDM). The calibrated model is then used to predict leaf area index (lai) based ...
7
votes
2answers
216 views
Find the closest enemy in a 2D grid with wrapping allowed
I'd like feedback on my solution to the outlined programming challenge. Is numpy a good candidate module for this? What might be a more efficient or Pythonic solution?
Closest Enemy II
Have ...
10
votes
2answers
1k views
Generating large testing datasets
I have made a small Python script that will generate some test sets for my project.
The script generates 2 datasets with the same dimensions n*m. One contains ...
1
vote
1answer
139 views
Vectorize pairwise edit distance computation [closed]
How can I vectorize this function? It takes an array of array of tuples as input and computes the distance between elements using Levenshtein's method. I found some simpler examples but they weren't ...
7
votes
0answers
146 views
Multithreaded HD Image Processing + Logistic reg. Classifier + Visualization
[I'm awaiting suggestions for improvement/optimization/more speed/general feedback ...]
This code takes a label and a folder path of subfolders as input that have certain labels ex: trees, cats with ...
4
votes
1answer
47 views
Move values to back of an np.array()
I'm trying to create the most efficient (not nicest/prettiest) way of moving particular values within an np.array() to the back of said array.
I currently have two different solutions:
...
1
vote
1answer
38 views
Optimization fo code snippet including multiple np.multipy statements
I working on optimizing some code for speed in a python module. I have pinned down the bottleneck and is a code snippet which calculates three np.ndarrays. Namely ...
8
votes
0answers
86 views
Calculation of independence test statistic
This code (~30lines) detects independence between variables in a dataset, by using a bootstrap of a statistic. This is fully-broadcasted, but it's still long.
Reading it, or making it run, do you ...
4
votes
2answers
97 views
First program in Python for data analysis
I'm just starting my adventure with python and I wanted to share my first project with you and ask for feedback and advice. This is a script for my friend to automate all calculations and database ...
4
votes
1answer
95 views
improving speed of this numpy-based diffraction calculator
I'm going to simulate diffraction patterns of a normal incident gaussian profile beam from a 2D array of point scatterers with a distribution in heights.
The 2D array of scatterer positions ...
2
votes
0answers
46 views
Generate Black and White confidence map masks for joints, same as OpenPose/Convolutional Pose Machines
Attached below is my code for generating the confidence maps for the joint locations, it is in the same vein as the maps generated for the paper Convolutional Pose Machines and OpenPose. I am ...
1
vote
1answer
71 views
