All Questions
Tagged with python-2.x performance
177 questions
4
votes
1
answer
1k
views
Pixelating black and white images
I am doing a function that pixelates black and white images. The function must be called pixelate(img, (pixel_width, pixel_height), mode).
The parameter ...
- 43
4
votes
2
answers
170
views
Gilbreath's conjecture in Python
I've written some code for testing Gilbreath's conjecture in python (2.x). The conjecture was shown (in the 90s) to be true up to n=1012. My code uses the primesieve module to quickly generate primes ...
- 141
6
votes
3
answers
993
views
7
votes
2
answers
150
views
9
votes
1
answer
4k
views
Optimizing a python script with large arrays
I have a python script that I'd like to speed up. I'm hoping some of the gurus here can offer some advice.
The script reads in some data from file: 2 arrays with lat/lon vals, 3 arrays with image ...
- 201
9
votes
3
answers
5k
views
Read one million records per second
The following code ingests 10k-20k records per second and I want to improve the performance of it. I'm reading json and ingesting it into the database using Kafka. I'm running it on the cluster of ...
7
votes
4
answers
282
views
Blocking production lines in Python
I have the following code that works great, but takes 40 mins to 1 hours to finish the task. It has at least 30 rules for blocking lines, and 30k+ orders to check. I'm new to Python (3 months working ...
2
votes
1
answer
781
views
Benchmark using floating point math
I wrote some code in Python, to benchmark mathematical operations. It does not do anything in particular besides just calculating a lot of floating point numbers.
To clarify: I need 128-bits ...
- 43
1
vote
2
answers
205
views
Updating ArcPy feature classes
I am looking for the way to speed up my script. The final output is as required, but it's clunky and really slow. I know it's slow because of the double for..in.. ...
- 121
6
votes
2
answers
362
views
Extract text from news articles
I have written a file to extract text from news articles. The program works as I want it to except it takes hours for it to write the output files. I have tried different buffering options as well as ...
- 185
7
votes
5
answers
1k
views
Project Euler #35 - Circular primes solution is slow
This is my code for project euler #35 (in Python):
The number, 197, is called a circular prime because all rotations of the digits: 197, 971, and 719, are themselves prime.
There are thirteen such ...
- 223
4
votes
2
answers
3k
views
Read firewall logs
I am still a Python beginner and would appreciate some help with this code.
I am looking through some firewall log files, more specific all lines with Deny in them. For those files I am extracting ...
- 43
4
votes
3
answers
162
views
Alberi puzzle creator - follow-up
This is a follow-up to this question.
Using his ExactCover class, I slightly modified Garth Rees' Alberi class to cope with ...
- 513
3
votes
2
answers
1k
views
Zeros in Factorial
Ram was busy calculating the factorials of some numbers. He saw a
pattern in the number of zeros in the end of the factorial. Let \$n\$ be
the number and \$Z(n)\$ be the number of zeros in the end ...
- 676
4
votes
2
answers
7k
views
Binary tree data structure
I am learning about the binary tree data structure and implemented some basic functionality.
It was working great with module level functions until I realized that I need to keep track of the root ...
- 5,127
2
votes
3
answers
449
views
List comprehension method
I have developed some Python code which revolves around two custom classes - a 'Library' class (Lib) which contains a Python list of several objects based on a 'Cas' class. I've not posted the code ...
- 163
4
votes
1
answer
4k
views
Find values in list which sum to a given value within tolerance
This is really a follow on from a question I asked earlier this year on Stack Overflow. I received a great answer to the following problem:
I'm trying to code up something simple and pythonic to ...
- 163
3
votes
2
answers
829
views
Project Euler #42 - Triangle Numbers
Here is the Euler problem referenced, it says:
The n\$^{th}\$ term of the sequence of triangle numbers is given by, t\$_n\$ =
½n(n+1); so the first ten triangle numbers are:
1, 3, 6, 10, 15, ...
- 641
7
votes
2
answers
991
views
Generating 3 combinations in Python
I have been experimenting around with generating all the n-combinations of an array. This code quickly generates all \$\text{k-combinations}\$ of a given array. I am testing my own implementation ...
- 5,127
5
votes
3
answers
1k
views
Will multi-threading or other method make my program run faster?
I didn't use multi-threading so far as I didn't need to. But as far as I've read, implementing them will make my program slightly faster than it actually is.
...
- 331
3
votes
2
answers
832
views
Generating a PANDAS DataFrame of simulated coin tosses
It's taking my machine quite a long time to execute 1 billion (1st loop x 10, 2nd loop x 1000, 3rd loop x 100,000) instructions. Suggestions for performance enhancements? Sources of potential concern:
...
- 147
5
votes
1
answer
189
views
Random Forest Code Optimization
I am new to Python. I have built a model with randomforest in python. But I think my code is not optimized. Please look into my code and suggest if I have deviated from best practices.
Overview about ...
- 53
2
votes
1
answer
638
views
Operating multiple columns of one pandas DataFrame using data from another
I have a DataFrame of data from a survey that was repeated over several years, asking people about their income and how much money they had in savings. For simplicity, let's pretend it looks like this:...
- 51
4
votes
4
answers
3k
views
Project Euler #8 - Largest product in series
The problem
The four adjacent digits in the 1000-digit number that have the greatest product are 9 × 9 × 8 × 9 = 5832.
...
- 5,725
4
votes
2
answers
2k
views
Splitting large text file and sorting by content
I have a large text file (~2GB) full of data. The data (sample below) gives an x, y, z coordinate, and a corresponding result on each line (there is other stuff but I don't care about it). The single ...
- 225
3
votes
3
answers
158
views
Calculating a pattern-match score on a ten-element array
I have a method in Python that is extremely slow:
...
- 31
4
votes
4
answers
3k
views
Efficiently concatenate substrings of long list of strings
I am having performance problems with the following python function:
...
- 151
4
votes
2
answers
227
views
Searching within multiple objects of the bucket
I have millions of files in the Google cloud storage's bucket. I want to search within files with a .index extension and retrieve the contents.
This is currently what I am doing, but the time ...
- 49
3
votes
1
answer
13k
views
Reading, writing and filtering a CSV file
I have a CSV with 5+ million rows and I want to filter it:
...
- 145
4
votes
1
answer
2k
views
Fast row-wise, parallel calculations on large, sparse matrix
I have a big (50k*150k) matrix with co-occurrences of nouns (rows) and adjectives (columns). As most nouns don't co-occurr with most adjectives, this matrix is very sparse, >99.9%, so I'm using the ...
- 121
4
votes
1
answer
10k
views
Sorting a list of names by greatest lexicographical order
This is a puzzle I found online that I have been trying to optimize and make the best version. I've tested it and it works great but I'm curious how it can be improved in any way.
Instructions:
"...
- 43
2
votes
2
answers
907
views
Project Euler problem 23 - non-abundant sums
I present my solution for Project Euler problem 23 written in Python (2.7).
The problem statement is:
A perfect number is a number for which the sum of its proper divisors
is exactly equal to ...
- 1,704
12
votes
3
answers
16k
views
Project Euler Problem 12 in Python
I'm trying to do Project Euler Problem 12, which reads as:
The sequence of triangle numbers is generated by adding the natural
numbers. So the 7th triangle number would be 1 + 2 + 3 + 4 + 5 + 6 + ...
- 223
3
votes
1
answer
363
views
Long running Python "server status" collector
I wrote a small program that collects the server status, caches it for a certain amount of seconds, and then sends that to a mothership server. It's broken down into a few pieces, starting with main....
- 2,814
7
votes
1
answer
2k
views
Optimizing critical loop for consuming a byte-buffer
I'm currently developing an Open Source project called sims3py using Python2.7. One overarching goal of the project is to ensure a Pure Python implementation of functions currently available only ...
- 275
4
votes
4
answers
655
views
Count the ways to partition an array into two equal sets
I need to count the number of ways to partition an array into two contiguous arrays such that those arrays contain exactly same set of items.
Suppose ...
- 143
6
votes
1
answer
1k
views
2
votes
1
answer
404
views
Python Koans multiplayer Greed Game
I finished the Python Koans because I wanted to practise Python and I completed the extra assignment which is the Greed Game (see the rules here).
I think there are no flaws but you never know.
<...
- 341
3
votes
3
answers
407
views
Python greed roll
I am doing the Python Koans to practise Python a bit more and I got to the greed dice project. It is a very simple dice game that scores a set of up to five rolls.
These are the rules:
...
- 341
13
votes
3
answers
7k
views
Optimizing an Anagram Solver
I've built an anagram solver in Python 2.7. It goes through a text file with over 100,000 words and checks if any permutations of the input matches the line. It works great, except it is very slow. I'...
- 287
3
votes
2
answers
2k
views
Process zip files
I have zip bundle, for example, abcd.zip, contains more zips like 1.zip, 2.zip etc. Inside of each child zip there is a .jpg file like 1.jpg, 2.jpg etc. There are so many other files but I need only ....
- 133
1
vote
1
answer
81
views
Write to a new, modified Excel file from an original one
I noticed that I've a HUGE bottleneck in this following function of mine. I can't see how to make it faster.
These are the profiling test results(keep in mind that I'm using a PyQt GUI so times can ...
- 133
4
votes
2
answers
5k
views
Twin Prime Algorithm Optimization
I have wrote the following algorithm to print out the Nth twin prime numbers:
...
- 141
1
vote
2
answers
273
views
Sieve of Erathosthenes speedup
While reading HackerNews, I learned about a data set of the first 50 million primes. This would be a lot, but his website has disclaimer:
n this directory I have the first fifty million primes in ...
- 242
2
votes
0
answers
2k
views
Optimize QuadTree to find K Nearest Neighbors
I'm looking a way to make my k nearest neighbors search more efficient. The context of the question is that I'm given a list of topics that have a unique ID (integer) and a (x,y) coordinate (floats) ...
- 113
4
votes
2
answers
2k
views
Custom sorting a list of objects efficiently
I'm trying to make sorting of objects in my program as fast as possible. I know that sorting with a cmp function is deprecated in python 3.x and that using keys is faster, but I'm not sure how to get ...
- 113
1
vote
2
answers
1k
views
Basic search engine
I want to improve efficiency of this search engine. It works in about 10 seconds for a search depth of 1, but 4 minutes at 2 etc.
I tried to give straightforward comments and variable names, any ...
- 111
3
votes
2
answers
378
views
Calculating the push or pull force of a molecule at a vertex
The basic idea of the code is to calculate the push or pull force at a vertex, given the number of "push causing molecules" and "pull causing molecules" at a polygon vertex.
The code thus mainly ...
- 399
3
votes
1
answer
701
views
3
votes
2
answers
208
views
Answer to an implementation of Project Euler #8
I saw that there was a previous question asking to review an answer to the Project Euler question #8. My answer didn't look anything like the answer in that question and as a beginner, I was wondering ...
- 143