Questions tagged [numpy]

NumPy is the fundamental package for scientific computing with the programming language Python.

Filter by
Sorted by
Tagged with
-3
votes
0answers
15 views

How to create a folders and subfolders in python in a tree like structure

I want to create folders and subfolders in the order: Test , Train ,valid. Each of these folders will contain 1 to n subjects and each subject will have three different folders. Ex: ...
0
votes
0answers
51 views

Count the number of neighbors

I have a numpy array called contour of \$N\$ coordinates (x,y) of dimension (N, 2). For each ...
1
vote
1answer
116 views

Ensuring performance of sketching/streaming algorithm (countSketch)

I have implemented what is know as a countSketch in python (page 17: https://arxiv.org/pdf/1411.4357.pdf) but my implementation is currently lacking in performance. The algorithm is to compute the ...
2
votes
3answers
2k views

Scaling and incrementing non-zero elements of a NumPy matrix

I have a NumPy matrix C and want create a copy of it cPrime, which has some operation of the original matrix to all non-zero ...
6
votes
0answers
32 views

Subtracting elements of datasets of an HDF5 file

I am trying to solve the following problem: Input: Input is two arrays (Nx4, sorted in column-2) stored in datasets-1 and 2 in HDF5 file (input.h5). N is huge (...
8
votes
2answers
30k views

K-means clustering in Python

The following code uses scikit-learn to carry out K-means clustering where \$K = 4\$, on an example related to wine marketing from the book DataSmart. That book uses excel but I wanted to learn Python ...
3
votes
1answer
80 views

Add values from a dictionary to subarrays using numpy

I'm working with numpy arrays and dictionaries, the keys of the dictionary are coordinates in the numpy array and the values of the dictionary are a list of values I need to add in those coordinates, ...
2
votes
2answers
80 views

Multiplying numpy arrays

I have written a function to multiply two numpy arrays. ...
1
vote
0answers
57 views

Find the available squares for rooks in empty board using boolean arrays

I will describe two techniques, then the question will be how I can make an algorithm that is more efficient, if possible. I want to find an efficient way to determine the squares where the pieces on ...
2
votes
0answers
15 views

Getting Graph and Block Animation to Run Smoothly Together

I have been working on some code that produces a live graphic of a graph and blocks moving. It works in theory but does not look like how I want. The animation runs very slow when showing both the ...
1
vote
1answer
89 views

Getting the square of the difference between many points in a NumPy array

I have an array with 40000 numbers that are floats or ints. I need to perform some calculation. To do this I have used nested for loop, but the code is really slow. Can I use something instead of ...
2
votes
1answer
37 views

Construct a numpy array by repeating a 1 dimensional array sliced at different indices

I have a pandas dataframe (called base_mortality) with 1 column and n rows, which is of the following form: ...
7
votes
3answers
170 views

Find the maximum price change over any 1 to 5 day rolling window

The code below solves the following task Find the maximum price change over any 1 to 5 day rolling window, over a 1000 day period. To be clear "any 1 to 5 day rolling window" means ...
2
votes
1answer
151 views

Smart Tic Tac Toe, a reinforcement learning approach

I'm currently familiarizing myself with reinforcement learning (RL). For convenience, instead of manually entering coordinates in the terminal, I created a very simple UI for testing trained agents ...
6
votes
0answers
56 views

Using get_dummies to create a Simple Recommender System - Cold Start

Question: was using get_dummies a good choice for converting categorical strings? I used get_dummies to convert categorical ...
4
votes
1answer
93 views

Generating unique 2D numpy arrays with 2 adjacent row elements swapped while using BFS in Python

So I have a 12 x 6 2D numpy array for input which consists of 7 possible board objects (characters from 'abcdefg'). From a 2D numpy array I wish to generate all the possible unique 2D arrays, in whose ...
5
votes
2answers
51 views

Analyzing the darkest pixels of an image in Python

This script takes an image copied from the clipboard and analyzes the n darkest pixels of the image. It will loop through each found value, prints out the value information and the quantity, then ...
2
votes
1answer
27 views

Get the last activity per user per day in a dataframe

I have many users. Each time a user uses their smartphone it register them. I am determining the last time each user used their smartphone each day. Additionally smartphone usages from 18:00 to 06:00, ...
1
vote
0answers
33 views

Calculation Correlation Time

I want to calculate the correlation time. First I calculate the auto-correlation function: $$ \begin{align} \chi(t) = &\frac{1}{t_{max}-t}\sum\limits_{t'=0}^{t_{max}-t}m(t')m(t'+t)\\ &-\frac{...
3
votes
1answer
39 views

Numpy array slicing/reshape/concatination

I'm sure my question exists on the internet, i just don't know the right formulations. I have a data-sample as input for a NN. This sample has the shape (1, 8, 28, 80). Basically it is 80 timesteps ...
0
votes
1answer
65 views

Image color-moment extractor [closed]

I was just wondering if there is a way to speed up the performances of this for loops in Python. I'm trying to process an image to get the color-moments without using libraries. It takes about 12sec ...
3
votes
2answers
63 views

Implementing a Steepest Descent Algorithm

I am teaching myself some coding, and as my first "big" project I tried implementing a Steepest Descent algorithm to minimize the Rosenbrock function: $$f(x, y) = 100 (y - x^2)^2 + (1 - x)^2$$ The ...
5
votes
1answer
164 views

Jupyter notebook style help + code suggestions for pandas

I wanted to open source some code to scrape and analyze publicly-filed stock buys and sells from U.S. senators. I'm not familiar with code style for Jupyter notebooks or pandas in general. Would it be ...
5
votes
3answers
246 views

Efficient numpy array manipulation to convert an identity matrix to a permutation matrix

Task I want to be able to generate the permutation matrix that splits a 1D array of consecutive numbers (i.e. even, odd, even, odd, even, odd, ...) into a 1D array where the first half are the evens, ...
7
votes
2answers
111 views

Largest product in a grid - Project Euler 11

A stab at Project Euler Problem 11: Largest product in a grid ...
6
votes
2answers
1k views

Calculating time deltas between rows in a Pandas dataframe

I am trying to compute the difference in timestamps and make a delta time column in a Pandas dataframe. This is the code I am currently using: ...
2
votes
0answers
25 views

Profiling for Bézier curve calculations

Recently I posted an answer on a question about Bézier curve calculations. As a micro-synopsis: there are three implementations of De Casteljau's algorithm here, including the original poster's, AJ ...
5
votes
1answer
35 views

Applying gaussian blur on RGBA images

I designed the code and it works quite well for images of lower res. However my program takes a lot of time and ram to display higher res images (occupies 2GB RAM for 4k images and takes 20 minutes). ...
3
votes
1answer
60 views

Fast way to calculate Standing Wave modes in NumPy

I am trying to plot the standing wave modes in a 2D box over time. The equation for this is: $$ A(x,y,t) = \sum\limits_{n,m} sin \left( \frac{n \pi x}{L_x} \right) sin \left( \frac{m \pi y}{L_y} \...
4
votes
1answer
67 views

Simulated annealing for magic square-of-squares puzzle

I'm working on a function that takes in a NumPy array containing only mutually distinct positive square numbers. I want the function to pick an element at random, find if there is a lower positive ...
6
votes
2answers
1k views

numpy performance of norm calculation using variable dimension

I am looking for advice to see if the following code performance could be further improved. This is and example using a 4x3 numpy 2d array: ...
4
votes
1answer
2k views

Better way to set number of threads used by NumPy

Background When NumPy is linked against multithreaded implementations of BLAS (like MKL or OpenBLAS), the computationally intensive parts of a program run on multiple cores (sometimes all cores) ...
3
votes
0answers
103 views

Computing Doppler delay on a meshgrid

Objective Plot the contour of the iso-doppler and iso-delay lines for a transmitter-receiver reflection on a specular plane. Implementation This Doppler shift can be expressed as follows: $$f_{D,0}(...
1
vote
0answers
19 views

How to optimize code to take advantage of NumPy broadcasting when calculating pairwise correlation?

I'm trying to take advantage of NumPy broadcasting and backend array computations to significantly speed up this function. Unfortunately, it doesn't scale so well so I'm hoping to greatly improve the ...
1
vote
1answer
29 views

What would be the computationally faster way to implement this 2D numerical integration?

I am interested in doing a 2D numerical integration. Right now I am using the scipy.integrate.dblquad but it is very slow. Please see the code below. My need is to ...
4
votes
1answer
73 views

Code for wireless communication work

I need some pointers on how I can speed up my code, as of now it is incredible slow for larger inputs. The way the code works is that the file Loc_Circle_50U.txt contains the true locations of 50 ...
6
votes
3answers
500 views

Improvement on data normalization

I have a part of code that is loading a dataset and normalizing property values to [0, 1]. My implementation is: ...
4
votes
1answer
105 views

Can this sampling code be vectorized or optimized?

I have some code that is slow because it loops over every row in an input matrix Y. Obviously, this code does not scale with the number of rows. I don't think it's ...
2
votes
1answer
109 views

Building optimal portfolios for all combinations of stocks

Hi I recently wrote some code in python that does the following: 1.) Pulls stock closing data from yahoo finance for x number of stock 2.) finds all possible combinations of x stocks in groups of y ...
13
votes
1answer
262 views

Population dynamic simulation on biological information maintenance 2

This question is the follow-up to this previous question. Background Using this simulation I investigate a system in which enzymes proliferate in cells. During the replications of enzymes, ...
5
votes
1answer
57 views

Binning whilst avoiding the need for for-loops in Python?

I am working on a relatively simple binning program, where I take a 5D array and bin it based on two 3D arrays to create a contour plot. See the sample code below. In actuality, my arrays are of size <...
6
votes
1answer
252 views

Dealing with lot of images and multiplications

With some basic knowledge of Python and referring a lot of sources, I have written the code below. But it takes half an hour for execution. How can I reduce the time? I read about vectorization but ...
4
votes
1answer
79 views

How to stop a Python script if the input values do not make sense within the environment in which they will be analyzed?

I am not a developer, but I needed to write a script. So I wrote it. Since I can't find an accessible tool that fulfills the same purpose, which I think may be helpful to others, I decided to ...
3
votes
0answers
65 views

Clean and fast MiniMax for numpy implementation of Tic-Tac-Toe

Tic-tac-toe is small enough to be solved completely but since I have a slow computer and this is an interesting exercise, I want to build the fastest possible minimax tic-tac-toe implementation in ...
9
votes
1answer
98 views

Find indices using for loop

I should estimate the effect of B fields on particle deflection using \$F=V \times B \times Force\$. I have a 2D mesh with \$n_x\$ and \$n_y\$ which show the number of points in each direction. \$B\$ ...
2
votes
1answer
176 views

AI In 2048 Game, Calculations so slow

Inspired by this answer on stackoverflow for AI implementation in 2048 game. I have been trying to make AI calculate best move by playing the game given specific times in memory preceding by one ...
2
votes
0answers
35 views

Convoluting 3D image with 2D

I have a single image of shape img.shape = (500, 439, 3) The convolution function is ...
-1
votes
1answer
43 views

Trying to get every 6th set of 4 bytes from a python array [closed]

I have an array of bytes 6144 bytes long. I want to extract a set of 4 bytes every 6th set of 4 bytes . So every 24 bytes I want to extract 4 of those bytes and join them in a new array. So ...
2
votes
1answer
56 views

Forecasting stock market data using Support Vector Regression

I coded this Support Vector Regression (SVR) myself following some equations in a journal (see here, or here (not in English)). The loss function used by the journal and the code below is mean ...
7
votes
2answers
636 views

Pad a ragged multidimensional array to rectangular shape

For some machine learning purpose, I need to work with sequences with different lengths. To be able to process efficiently those sequences, I need to process them in batches of size ...

1
2 3 4 5
13