Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
16
votes
1answer
36k views

Efficiently filter a large (100gb+) csv file (v3)

I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this question. The problem is as follows: I need to filter ...
2
votes
1answer
867 views

Efficiently filter a large (100gb+) csv file (v1)

EDIT: This question was followed up by this question, with improved code. I have a set of pretty large files (>2 GB, over 30m rows) containing intraday data in the following format: ...
7
votes
3answers
5k views

Compute mean, variance and standard deviation of CSV number file

I've written my first C++ application this week and I wanted to see if there's anything that aren't up to standards or better ways to do what I'm currently doing. For the goal of the exercise, let's ...
7
votes
1answer
11k views

Printing out JSON data from Twitter as a CSV

I'm extremely new to Python, and the Twitter API, but I found an example online that walked me through the process. Now that I've been playing around with it for awhile, I've begun to push the limits ...
6
votes
2answers
1k views

Efficiently filter a large (100gb+) csv file (v2)

EDIT: This question is followed up by this question. I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this ...
19
votes
4answers
8k views

Web scraping the titles and descriptions of trending YouTube videos

This scrapes the titles and descriptions of trending YouTube videos and writes them to a CSV file. What improvements can I make? ...
12
votes
1answer
864 views

Find the peak stock price for each company from CSV data

During the hiring process, a company gave me this test: Q) Consider Share prices for a N number of companies given for each month since year 1990 in a CSV file. Format of the file is as below with ...
10
votes
1answer
52k views

Writing lists to .csv file

I'm teaching myself programming, using Python as my initial weapon of choice. I have learnt a few basics and decided to set myself the challenge of asking the user for a list of names, adding the ...
8
votes
3answers
450 views

Calculating effective shop inventories from CSV files

Program purpose The purpose for the program is to create a file containing the effective inventory for each stocked item stored in a collection of shops. Input and algorithm details For clarity, ...
7
votes
3answers
366 views

A command line Java program for computing grade point average

(See the next iteration.) I have this command line program for computing GPA (grade point average). Given credit sequence \$\langle c_1, c_2, \dots, c_n \rangle\$ and grade sequence \$\langle g_1, ...
6
votes
1answer
661 views

Interface to a CSV dinosaur database

I have just finished with the first Ruby exercise on Level Up Rails, and wanted to get an idea on how I can refactor the code. You can find the original exercise on github - including the ...
6
votes
2answers
1k views

Calculate food company sales for the year

I am working on a Python project where a food processing company is trying to calculate its total sales for the year. Python has to read from a text file where its divided into for categories split by ...
3
votes
2answers
170 views

Validating a CSV list of contacts and convert it to JSON

I've written a class that takes a file, validates the formatting of the lines from an input file and writes the set of valid lines to an output file. Each line of the file should have a first name, ...
2
votes
5answers
6k views

CSV reader and parser

I have implemented a CSV reader. I think I did pretty well. Since CSV is a loosely defined format to begin with I decided to allow some malformations, like anything but a delimiter after an enclosed ...
9
votes
3answers
233 views

Collate votes on MP3 files to list them by popularity

For my first Python project, I decided to create a script that would automatically collate MP3 files into a music countdown. I wanted to 'learn by doing', so I dived straight in and used the Python ...
8
votes
1answer
611 views

Outputting a .txt file from a .csv file

I wrote this little script in PHP a bit ago. It takes a .csv file and outputs a .txt file with certain data arranged in a "psuedo-array" looking format. I recently started messing around with Ruby so ...
8
votes
3answers
175 views

MP3 Vote Collator - Final

Here is the completed code for my first Python project, for which I received some help and feedback here: Collate votes on MP3 files to list them by popularity Seeing as there have been considerable ...
7
votes
4answers
264 views

Blocking production lines in Python

I have the following code that works great, but takes 40 mins to 1 hours to finish the task. It has at least 30 rules for blocking lines, and 30k+ orders to check. I'm new to Python (3 months working ...
7
votes
1answer
2k views

Efficiently adding huge amounts of data from CSV files into an SQLite DB in Java

I'm trying to parse values from a CSV file to a SQLite DB, however the file is quite large (~2,500,000 lines). I ran my program for a few hours, printing where it was up to, but by my calculation, the ...
6
votes
1answer
909 views

CSV writer implementation

I've recently been assigned to write up a CSV writer with "as much flexibility" as possible concerning pretty much all (output stream, string fields with potential new line characters, separator ...
5
votes
1answer
159 views

Batch upload users to nextcloud on docker with bash csv

I have this bash script that adds users to NextCloud from a csv file. It expects to be run from the same directory as the docker-compose.yml file. I'm pretty new to bash scripting and I'm wondering ...
4
votes
3answers
298 views

A program to display, update and save a dictionary as .csv

I'm now looking for feedback on v2.0 of this program instead. I'd like some feedback about my code. What bad habits do I have? What advice could help me write more Pythonic? I'm trying to write a ...
4
votes
2answers
130 views

Dataset and frequency list generation by looping over files

I have been told that it would be wise to split up my code into semantically useful blocks. I tried the following but I need feedback. What I already did: As you can see I create two .csv files, so ...
3
votes
1answer
210 views

A module to display, update and save a dictionary as .csv v2.0

I'd like feedback on my revised code after my previous question. The goal is to create a module which can display, update, save dictionaries as .csv files. I'll use this in another program to update ...
3
votes
1answer
204 views

Parsing positive integers from CSV file

I Needed to parse some positive integers with the following constraints: If possible use C++ only (no C API) Needs to be able to process parsed data after each newline Should be reasonably fast ...
2
votes
2answers
1k views

Comparing columns from two CSV files - follow-up

This is the second part of a question, you can find the first part here: Comparing columns from two CSV files I've made some changes to the script and here is what it looks like now: ...
2
votes
2answers
5k views

Creating a tuple from a CSV file

I have written code that reads in a CSV file and creates a tuple from all the items in a group. The group ID is in column 1 of the table and the item name is in column 2. The actual datafile is ~500M ...
2
votes
0answers
111 views

Building nested object structures from CSV input

I'm reading big csv files and mashing the lines into hierarchical objects for analysis and display. The csv might look like this: ...
2
votes
2answers
822 views

Speeding up and fixing phone numbers from CSVs with Regex

I've hodgepodged together an attempt to extract all phone numbers from all CSVs in a directory, regardless of where they are and what format they're in. I want all phone numbers to be printed to a ...
2
votes
1answer
274 views

CSV writer implementation refactored

Based on my initial implementation found here and the suggested improvements in answers and comments, I've refactored the code. The main changes are: Removed redundant buffering. Simplified ...
1
vote
1answer
2k views

Comparing columns from two CSV files

This question has a second part here: Comparing columns from two CSV files - follow-up I'm currently trying to compare two CSV files column by column but only when their indexes match. Here is what ...
1
vote
1answer
132 views

Streaming learning OCaml

I wrote a simple online logistic regression, calibrated using gradient descent to compare the speed of an OCaml implementation vs the same Python script, executed with Pypy. It turned out that the ...
1
vote
2answers
833 views

Student administration application using a .csv file for persistence

The application is a student administration system using a .csv file for persistence as part of our school project. This is my first created a basic DAO. I'd like to have some feedback about what ...
1
vote
1answer
2k views

Check that a data file contains the expected number of rows and columns of integers

This is a class I wrote check to check the format of .dat files. My code makes sure that the content of the files satisfies the conditions. A file can be deemed invalid if: The file can't be found. ...
1
vote
0answers
322 views

A command line Java program for computing grade point average - follow-up

(See the previous and initial iteration.) I have refactored the API based on excellent review by cbojar. Also, a honorary mention goes to janos for pruning some unnecessary code out of the transcript ...
-4
votes
3answers
7k views

Write object data to CSV files [closed]

Does this method follow the rules of objective programming? If not, how can I change it? ...