All Questions
Tagged with python-2.x performance
177 questions
24
votes
4
answers
485
views
When it rains, it pours - August 2016 Community Challenge
1. Introduction
This code is my attempt at solving the August 2016 Community Challenge. Coming from a city where it rains cats and dogs on a daily basis this challenge was right up my alley =)
2. ...
- 5,725
23
votes
6
answers
13k
views
Check if a Sudoku board is filled out correctly
The code below takes a filled out Sudoku board of size NxN, with sub-blocks of nxn, and checks if the solution is correct.
main_function takes a board as input. ...
- 2,067
22
votes
4
answers
10k
views
Regex to parse semicolon-delimited fields is too slow
I have a file with just 3500 lines like these:
filecontent= "13P397;Fotostuff;t;IBM;IBM lalala 123|IBM lalala 1234;28.000 things;;IBMlalala123|IBMlalala1234"
...
- 223
20
votes
1
answer
302
views
Rainfall - August 2016 Community Challenge (Hooray for graphs!)
A solution for the August 2016 Community Challenge (Rainfall).
I've intentionally left out error checking on the file formatting.
This seems like a very obvious graph problem to me - we want to ...
- 7,854
14
votes
1
answer
10k
views
Faster computation of barycentric coordinates for many points
I'm just starting to understand the Python syntax and I created a module that does what I wanted, but really slow.
Here are the stats of cProfile, top 10 ordered by ...
- 475
14
votes
1
answer
231
views
Download images from an image board based on tags and blacklists
This code is meant to download images from an image board to create a local backup. Images are grouped by tags selected by the user and then downloaded using multiprocessing. Normal use of this ...
- 241
13
votes
3
answers
7k
views
Optimizing an Anagram Solver
I've built an anagram solver in Python 2.7. It goes through a text file with over 100,000 words and checks if any permutations of the input matches the line. It works great, except it is very slow. I'...
- 287
12
votes
3
answers
2k
views
Finding the prime factors of a number in Python 2
I’ve created a function that takes a number and, if it’s prime, tells you so, or if it’s composite, gives you the prime factors of the number (and if it’s 1, tells you that it’s neither).
...
- 223
12
votes
3
answers
16k
views
Project Euler Problem 12 in Python
I'm trying to do Project Euler Problem 12, which reads as:
The sequence of triangle numbers is generated by adding the natural
numbers. So the 7th triangle number would be 1 + 2 + 3 + 4 + 5 + 6 + ...
- 223
12
votes
1
answer
4k
views
A CNN in Python WITHOUT frameworks
Here's some code that I've written for implementing a Convolutional Neural Network for recognising handwritten digits from the MNIST dataset over the last two days (after a lot of research into ...
- 395
12
votes
1
answer
168
views
Create edges from cells
I have a (large) number of nodes and (triangular) cells, e.g.,
cells_nodes = [
[0, 3, 1],
[0, 2, 3]
]
This example represents the small mesh
(Please ...
- 221
11
votes
3
answers
4k
views
Removing entries from a dictionary containing bad words
I have a dictionary with each item containing a quote in another dictionary:
...
- 305
11
votes
2
answers
817
views
"FIZZ BANG BUZZ!" 3,7,11 efficiency
I wrote a fizz buzz variation method in Python which prints "fizz", "bang", and "buzz" for 3, 7, and 11 respectively.
...
- 1,499
11
votes
1
answer
1k
views
Poor man's f-strings in Python 2.6
f-strings is a new feature in the upcoming python release which basically lets you do this:
>>> s = 'qwerty'
>>> f'{s} {1+2}'
'qwerty 3'
Not ...
- 213
10
votes
4
answers
14k
views
Consolidate list of ranges that overlap
I wanted to implemented an algorithm in Python 2.4 (so the odd construction for the conditional assignment) where given a list of ranges, the function returns another list with all ranges that overlap ...
- 228
10
votes
1
answer
850
views
Decoding custom packets with Python using Twisted
This server require fast decoding of packets from clients, as I am expecting 5000 concurrent clients that is sending packets possibly every second. This is a game server (5000 players walking send ...
- 295
9
votes
3
answers
5k
views
Read one million records per second
The following code ingests 10k-20k records per second and I want to improve the performance of it. I'm reading json and ingesting it into the database using Kafka. I'm running it on the cluster of ...
9
votes
3
answers
2k
views
Script for finding cheating students in a quiz
Below is a script to find cheating students in a quiz in a daily moodle activity log exported in .xls. It works fine and is written in a procedural way.
Essentially the script isolates the activity ...
- 155
9
votes
2
answers
300
views
Simulating surname convergence in a population
I was curious a few weeks ago about how surnames come and go (mostly on the go). I wanted to know how quickly surnames die out, because (in my simplified simulation world) once the last person with a ...
- 255
9
votes
1
answer
9k
views
Find the nearest point of a given set of points
Suppose there are a set of given points (represented by x and y two dimensional coordinates), and for any given point A, I want ...
- 3,453
9
votes
1
answer
4k
views
Optimizing a python script with large arrays
I have a python script that I'd like to speed up. I'm hoping some of the gurus here can offer some advice.
The script reads in some data from file: 2 arrays with lat/lon vals, 3 arrays with image ...
- 201
8
votes
2
answers
103
views
Comparing Server Version Number History
I have this code that compares all version numbers of servers. (The version numbers are stored in an XML file) Before the code would compare the current version number with the previous. This caused ...
- 183
8
votes
3
answers
2k
views
Replacing English words in a string with their Spanish equivalents by dictionary lookup
I want to improve my code's performance and running time, looking for write my loops better.
For example, I have a dictionary that contains words as keys, and their translation in Spanish as values.
...
- 81
8
votes
2
answers
14k
views
Python code that compares two files byte by byte
I wrote a Python (2.7) script that compares a file byte by byte. filecmp was not suitable for the application because it compares metadata as well. How can I ...
- 275
8
votes
2
answers
749
views
Sum of prime factors of binomial coefficients over 9000!\${}\$
The problem
I have lately been working Project Euler: 231:
The prime factorisation of binomial coefficients
The binomial coefficient \$ ^{10}C_3 = 120 \$.
\$ 120 = 2^3 × 3 × 5 = 2 × 2 ×...
- 5,725
8
votes
1
answer
401
views
Kernel Convolution in Python 2.7
The code I wrote performs a mean blur on an image (I hardcoded it as zebra.jpg for testing purposes).
My problem is that for an image of 39KB image it take minutes to perform, is there any way of ...
- 81
8
votes
2
answers
154
views
7
votes
5
answers
1k
views
Project Euler #35 - Circular primes solution is slow
This is my code for project euler #35 (in Python):
The number, 197, is called a circular prime because all rotations of the digits: 197, 971, and 719, are themselves prime.
There are thirteen such ...
- 223
7
votes
4
answers
282
views
Blocking production lines in Python
I have the following code that works great, but takes 40 mins to 1 hours to finish the task. It has at least 30 rules for blocking lines, and 30k+ orders to check. I'm new to Python (3 months working ...
7
votes
2
answers
2k
views
Program to find magic square of squares
I have a program which finds, well, a magic square of squares. The problem is, it's quite slow - it processes around 50 numbers in half a second when the number range is above 40,000.
Is there any ...
- 273
7
votes
2
answers
991
views
Generating 3 combinations in Python
I have been experimenting around with generating all the n-combinations of an array. This code quickly generates all \$\text{k-combinations}\$ of a given array. I am testing my own implementation ...
- 5,127
7
votes
2
answers
150
views
7
votes
3
answers
3k
views
Finding the fastest common prefix of 2 strings in Python
I'm comparing my algorithms, slice_prefix and bit_prefix, with existing ones to find the common prefix length of 2 strings as ...
7
votes
2
answers
5k
views
Optimizing the speed of a web scraper
I have just written this code to scrape some data from a website. In its current state it works fine, however, going by my tests on the script, I discovered that with the amount of data I am ...
- 227
7
votes
1
answer
2k
views
Optimizing critical loop for consuming a byte-buffer
I'm currently developing an Open Source project called sims3py using Python2.7. One overarching goal of the project is to ensure a Pure Python implementation of functions currently available only ...
- 275
7
votes
1
answer
1k
views
Plotting terrain pixels with PyGame based on random NumPy array
I am experimenting with Perlin Noise and random map generation. I have a 2D numpy ndarray full of 16-bit floats called map_list that I call from the singleton ...
- 421
6
votes
3
answers
993
views
6
votes
2
answers
6k
views
Processing large file in Python
I have some code that calculates the "sentiment" of a Tweet.
The task starts with an AFINN file, that is a tab-separated list of around 2500 key-value pairs. I read this into a ...
- 1,119
6
votes
3
answers
286
views
Class scheduling tool with image output
I have a multilayered dictionary that contains information about classes. I am using this to code an automatic schedule builder that I will eventually add to a separate Tkinter application I made that ...
- 231
6
votes
2
answers
854
views
Algorithm to say if win or lose using player's rating and randomness
I wrote a small simulator to understand the Elo rating system. I can create players, matches and tournaments, and I want to be able to predict the match ending depending on the rating of each player ...
- 639
6
votes
2
answers
362
views
Extract text from news articles
I have written a file to extract text from news articles. The program works as I want it to except it takes hours for it to write the output files. I have tried different buffering options as well as ...
- 185
6
votes
1
answer
886
views
Palindromes that are sum of consecutive squares
I have been working on a Project Euler: 125, which took me ages to solve. The problem and source are cited below
The palindromic number 595 is interesting because it can be written as
the sum of ...
- 5,725
6
votes
1
answer
1k
views
6
votes
1
answer
488
views
Python knapsack program
I have this data for the knapsack problem:
List of product prices = MENU
Most popular product count = MOST_POPULAR_COCKTAIL
...
- 109
6
votes
1
answer
437
views
Add an edge to a graph to minimize the average shortest path length
This program is used to find the nodes in a grid network, between which, if an edge is added, the average shortest path length of the entire grid reduces by the most.
"Average shortest path length" ...
- 61
6
votes
1
answer
223
views
Improving the performance of a webscraper
I have here a modified version of a web scraping code I wrote some weeks back. With some help from this forum, this modified version is faster (at 4secs per iteration) than the earlier version. ...
- 227
6
votes
1
answer
312
views
Increase the speed of this Caesar Cipher
Here's my attempt at a Caesar Cipher encoder/decoder.
If given a key, it will encrypt the given string. However, if you do not specify a key, it checks each of the 26 possible keys and returns the ...
user51723
6
votes
3
answers
2k
views
5
votes
3
answers
1k
views
Will multi-threading or other method make my program run faster?
I didn't use multi-threading so far as I didn't need to. But as far as I've read, implementing them will make my program slightly faster than it actually is.
...
- 331
5
votes
1
answer
839
views
Counting the number of elements in a sorted array
Suppose I have a sorted array, and each element may appear multiple times.
To make it simple, here is an example:
...
- 3,453