Skip to main content
60 votes

Python web-scraper to download table of transistor counts from Wikipedia

Is there a way to simplify this code? Yes. Don't scrape Wikipedia. Your first thought before "should I need to scrape this thing?" should be "Is there an API that can give me the data I want?" In ...
Reinderien's user avatar
53 votes
Accepted

Counting rows in a CSV file that correspond to a database row, each with a million records

Your inner loop appears to be unnecessary. Why not create a lookup: ...
RobH's user avatar
  • 16.9k
51 votes

Python web-scraper to download table of transistor counts from Wikipedia

Let me tell you about IMPORTHTML()... So here's all the code you need in Google Sheets: ...
Eric Duminil's user avatar
  • 3,960
41 votes

Country code lookup for each line in a CSV file

Your loop is close to 200 lines, that's waaaay too long. That's long for a full class, let alone one method, let alone part of one method. Why do you use ...
BCdotWEB's user avatar
  • 11.2k
41 votes

Web scraping the titles and descriptions of trending YouTube videos

Why web-scrape, when you can get the data properly through the YouTube Data API, requesting the mostpopular list of videos? If you make a ...
200_success's user avatar
18 votes
Accepted

Reading a repetitive file with repetitive code

Well, you could find a "Pythonic" way to skip around a file, but that still wouldn't make it good code, in my opinion. There are some fundamental issues with your approach that make it rather hackish:...
200_success's user avatar
18 votes
Accepted

Parsing RFC 4180 CSV with GOTOs

1) I would save constants (doubleQuote, etc.) as fields, so they don't take up extra space in already fairly large method body. 2) I think your use of ...
Nikita B's user avatar
  • 13k
18 votes
Accepted

System that manages employee data for managers

Welcome to Code Review! Kudos to writing a fairly large program. Several things pop-out from your program. But, a few things first. If you are using any intelligent editor; please see if you can get ...
hjpotter92's user avatar
  • 8,716
17 votes

Country code lookup for each line in a CSV file

Instead of using linq lookups for the lists you should map your data into dictionaries, which are MUCH faster to access specific instance of your data, memorywise you might take a hit but the mapping ...
Janne Matikainen's user avatar
17 votes

Sensor logger for Raspberry Pi in a stratospheric probe

Do not call main recursively. You are setting yourself up for stack overflow. Consider instead ...
vnp's user avatar
  • 56.4k
16 votes
Accepted

Reading large amount of player properties from CSV

The first thing I would do is break that up somehow. Define a struct to encapsulate the three common properties between each stat: ...
Der Kommissar's user avatar
16 votes

Python web-scraper to download table of transistor counts from Wikipedia

Make sure to follow naming conventions. You name two variables inappropriately: ...
Carcigenicate's user avatar
15 votes

Web scraping the titles and descriptions of trending YouTube videos

Context manager You open a file at the beginning of the program and close it explicitly at the end. Python provides a nice way to allocate and release resources (such as files) easily: they are ...
SylvainD's user avatar
  • 29.2k
14 votes

Counting rows in a CSV file that correspond to a database row, each with a million records

The code appears to be broken, you're not break;ing after finding a match so all records will likely have row[12] == "N". You ...
Johnbot's user avatar
  • 2,974
14 votes

System that manages employee data for managers

Don't do wildcard imports Use import tkinter as tk and then prefix all tk classes and commands with tk. (eg: ...
Bryan Oakley's user avatar
  • 2,089
14 votes

Read CSV and use bidirectional BFS to find the shortest connections between actors

It looks like you found a pretty nice approach there already. A few general things stand out: Using globals Magical constants and unsafe C-style arrays/raw pointers: ...
sehe's user avatar
  • 1,322
13 votes
Accepted

Aggregate prescriptions per drug in a CSV file

Performance Your performance problem is primarily due to the duplicate_id() function, namely the if id in id_list test. ...
200_success's user avatar
12 votes

Parsing RFC 4180 CSV with GOTOs

The biggest problem with gotos is that they make it really hard to refactor code easily. Your loop body alone is 66 lines of code long and has a nesting depth of 5. Length of code and nesting depth ...
Voo's user avatar
  • 525
12 votes

Are there ways to speed up this string fuzzy matching in Golang?

Review I - package go-fuzzywuzzy After a quick read of your Go code, the Go code for package go-fuzzywuzzy, and the Go ...
peterSO's user avatar
  • 3,466
11 votes
Accepted

CSV file reader in PHP that supports large files (>15k lines)

Performance As performance is your main concern, let's face this first. To complete the example CSV-file with ~36k lines your original script needs around 139s*. The main bottlenecks are ...
insertusernamehere's user avatar
11 votes
Accepted

Sensor logger for Raspberry Pi in a stratospheric probe

Have you already executed the code to see how it performs and if the battery will last? There is that famous Donald Knuth quote saying premature optimization is the root of all evil (or at least most ...
Kim's user avatar
  • 226
10 votes
Accepted

Python search for array in large text file

Performance It should be possible to accomplish this task in seconds rather than minutes, with the right data structure. This is your main mistake: ...
200_success's user avatar
10 votes

Country code lookup for each line in a CSV file

if (values.Length < 3) { return; } if (values.Length >= 2) { How should in the second if the ...
Heslacher's user avatar
  • 50.4k
10 votes
Accepted

Searching for data from file1 in file2

There are things to improve: when you use with context manager, you don't need to close the file explicitly you can combine two context managers into one reducing ...
alecxe's user avatar
  • 17.3k
10 votes

Web scraping the titles and descriptions of trending YouTube videos

I'd definitely look into using an API directly as @200_success suggested to avoid any web-scraping or HTML parsing, but here are some additional suggestions to improve your current code focused mostly ...
alecxe's user avatar
  • 17.3k
10 votes
Accepted

Reducing the amount of duplicated code (python) - cricket matches

Disclaimer: I know nothing whatsoever about the rules of cricket. Useless check Because either info['toss_decision'] equals ...
SylvainD's user avatar
  • 29.2k
9 votes
Accepted

Calculating a table of deBroglie wavelengths for various electron energies

First, simplify: factor out \$C^4\$ out of square root: \$\lambda(E_k) = h\left/C\sqrt{(\frac{E_k}{C^2}+m_e)^2-m_e^2}\right.\$ Simplify even more: factor out \$m_e^2\$: \$\lambda(E_k) = h\left/C ...
vnp's user avatar
  • 56.4k
9 votes
Accepted

Convert XML to CSV

First of all, I'd split the convert method out into it's own thing - separate from the loading and saving: ...
Pj-'s user avatar
  • 106
9 votes

Parse one table out of CSV with multiple titled tables with the Python CSV module

In general this code look very nice and seems to follow style guidelines well. I found your code easy to follow. You asked for: ... suggestions to make the code more compact, idiomatic, and fit ...
Stephen Rauch's user avatar
9 votes
Accepted

Transform dict of lists to CSV

As per PEP 8, the standard indentation for Python code is 4 spaces. Since whitespace is significant in Python, this is a pretty strong convention that you should follow. The code itself isn't bad, ...
200_success's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible