Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
37
votes
1answer
1k views

Utility that decodes and logs UDP packets

I have written the following utility, as my first non-tutorial program in Go. The purpose of the utility is to connect to a torque/force sensor (aka load-cell) via UDP; to send an initialization ...
30
votes
6answers
6k views

Counting rows in a CSV file that correspond to a database row, each with a million records

I have two DataTables: dt: is populated from a CSV file with over 1.7 million rows dataStructure.Tables["AccountData"]: is ...
26
votes
5answers
3k views

Python web-scraper to download table of transistor counts from Wikipedia

I have been looking for answers for how to easily scrape data from Wikipedia into a CSV file with Beautiful Soup. This is the code so far. Is there an easier way to do it? ...
23
votes
5answers
2k views

Inventorizing the universe

I recently started re-factoring a project I hadn't touched in many months. The original purpose was to be an inventory management system for a game I played, Ogame. In this game you play inside a ...
21
votes
4answers
7k views

Regex to parse semicolon-delimited fields is too slow

I have a file with just 3500 lines like these: filecontent= "13P397;Fotostuff;t;IBM;IBM lalala 123|IBM lalala 1234;28.000 things;;IBMlalala123|IBMlalala1234" ...
21
votes
5answers
21k views

Parsing CSV to specific format

I've written this code to read CSV files written in a specific format. I would like to gather some feedback on where it could be improved. I'm trying to get into the Test-Driven-Development ideology ...
21
votes
5answers
2k views

Generating CSV strings for various 3rd party utilities

I'm generating CSV strings for various 3rd party utilities and this section of code gets repeated in many classes. Is there a better way to generate this string? ...
20
votes
4answers
6k views

System that manages employee data for managers

This code is for an employee management system. The goal that I am trying to reach is to allow managers to access data of their employees more easily. The data that is entered is saved to a CSV file ...
19
votes
4answers
5k views

Web scraping the titles and descriptions of trending YouTube videos

This scrapes the titles and descriptions of trending YouTube videos and writes them to a CSV file. What improvements can I make? ...
19
votes
8answers
49k views

Java function to read a CSV file

I have a java function that reads a csv and returns its content as a Vector<Vector<String>>. The function seems to work as I need it, but my gut ...
19
votes
2answers
1k views

API for looking up Japanese postal code

Most of my experience is in Python, Java and embedded C, and I need to learn Scala and Play for a new job. As an exercise, I decided to create a very simple API for looking up area by postal code, ...
17
votes
4answers
1k views

Sensor logger for Raspberry Pi in a stratospheric probe

I'm writing a Python Script for a Raspberry Pi to measure different sensors. We are planning to send the Pi with that Script running to the stratosphere, so the power usage for the Pi is limited. I ...
16
votes
1answer
33k views

Efficiently filter a large (100gb+) csv file (v3)

I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this question. The problem is as follows: I need to filter ...
12
votes
3answers
1k views

Retrieving a csv header

I have two seperate calls that returns a .csv file. I need to extract a couple of headers from the files depending on which one is called. The headers does not always sit in the same index, and the <...
12
votes
2answers
21k views

CSV reader using StreamReader and LINQ

Mostly due to readability, I am using the following code to find a specific csv file on a drive read that CSV file using a streamReader parse it into a ...
12
votes
1answer
833 views

Find the peak stock price for each company from CSV data

During the hiring process, a company gave me this test: Q) Consider Share prices for a N number of companies given for each month since year 1990 in a CSV file. Format of the file is as below with ...
11
votes
4answers
2k views

Country code lookup for each line in a CSV file

I'm trying to write a program that has a for loop, inside of which I have lambda expression to get a certain value. The loop has about 20,000 iterations. Including ...
11
votes
4answers
1k views

Parsing RFC 4180 CSV with GOTOs

One of my data-import tools needs to support CSV files. I thought that parsing CSV is such a simple task that I didn't want to use any any external libraries for that. So here is one more RFC 4180 CSV ...
11
votes
3answers
836 views

Aggregate prescriptions per drug in a CSV file

I have a question that is similar in spirit to this previously asked question. Nonetheless, I can't seem to figure out a suitable solution. Input: I have CSV data that looks like (FICTIONAL ...
11
votes
3answers
8k views

Convert all CSV files in a given directory to JSON using Python

I am attempting to convert all files with the csv extension in a given directory to json with this python script. I am wondering if there is a better and more efficient way to do this? Here is my ...
11
votes
3answers
435 views

Print Student Names from CSV

Description Print all student names from a CSV file. CSV Format: CSV file does not contain string delimited by double quotes. None of the cells has "," character ...
10
votes
2answers
1k views

Reading a repetitive file with repetitive code

I made this code for take some data from a .txt file. The text file is a large list of data from temperature with a format that is repetitive and have some lines at the beginning with the station ...
10
votes
2answers
1k views

Read/write a pipe-delimited file line by line with some simple text manipulation

This code that I wrote is supposed to read/write a pipe-delimited file line by line to a new file with some simple text manipulation. (It also adds two new columns) and publishes a "Status Update" ...
10
votes
2answers
649 views

Searching for data from file1 in file2

In order to answer a given question on Ask Ubuntu I came up with two scripts for doing the following task: Compare two files for an occurrence of column no. 6 value in file1 occurs in file2. On ...
10
votes
2answers
438 views

Averaging numbers from a file, obeying Single Responsibility Principle

Task: Given comma_separated.txt file with string: 1,2,3,4,5,6,7,8,9,10,11,12...,n Calculate the average ...
10
votes
2answers
6k views

Parsing CSV file with BufferedReader vs Scanner

I have to parse a file line-by-line and in single line I have split by ",". The first string is the name and the second is the count. Finally, I have to display the Key and Count, for example: ...
10
votes
1answer
7k views

Faster Django CSV generation for several million database entries

I am writing a CSV file generator that's filtering through about seven million database entries (MySQL backend). This part is especially slow and I was wondering if there is a way to make it much ...
10
votes
1answer
32k views

Writing lists to .csv file

I'm teaching myself programming, using Python as my initial weapon of choice. I have learnt a few basics and decided to set myself the challenge of asking the user for a list of names, adding the ...
10
votes
2answers
424 views

In memory data access for CSV files that may have changed

I'm reading data from CSV files and store the result in a simple DataContext instance that gets injected throughout my application. ...
10
votes
1answer
271 views

Loading text into an array when fields are in random order

This script reads records from a CSV file of customer names and addresses. The first record of the file is the header (field names); the fields could be in any order, and sometimes have different ...
9
votes
5answers
2k views

Convert total minutes into “Hours:minute:seconds” for marathon data

The point of the script below is to read in the 10th column which is the total number of minutes a runner took to finish the marathon and convert it into form of "hours:minutes:seconds". Is there a ...
9
votes
3answers
229 views

Collate votes on MP3 files to list them by popularity

For my first Python project, I decided to create a script that would automatically collate MP3 files into a music countdown. I wanted to 'learn by doing', so I dived straight in and used the Python ...
9
votes
2answers
3k views

Getting lists of values from a CSV

I have a function that takes a column title, and a response.body from a urllib GET (I already know the body contains text/csv), and iterates through the data to build a list of values to be returned. ...
9
votes
1answer
111 views

Text menu to browse CSV data about seal species

I am writing in C++, I have completed a small program which parses CSV files from IUCN and presents the species data using menus. I used a lot of loops. It works, however it is buggy when processing ...
8
votes
4answers
5k views

Reading a text file that contains several CSV-like tables

I wrote some extension methods to read CSV-styled text directly into a datatable or dataset or write them to this format. Is it right to use the methods as extensions or should I create a separate ...
8
votes
4answers
8k views

Convert XML to CSV

I'm pretty sure this code can be optimized, but I'm not talented enough in Linq to do it myself. Here's what I'm trying to do: I have an XML file that needs to be converted into a .csv file. The XML ...
8
votes
3answers
3k views

Parsing Excel file into CSV file

EDIT: Thanks everyone for your answers, this is moving quickly now and it's always essential to get better with design and best practice! I am loading in an excel file to a dataset using ...
8
votes
1answer
562 views

Outputting a .txt file from a .csv file

I wrote this little script in PHP a bit ago. It takes a .csv file and outputs a .txt file with certain data arranged in a "psuedo-array" looking format. I recently started messing around with Ruby so ...
8
votes
2answers
93 views

Iterate list to map entries in Python

I have two files, namely: File1: CL1 AA XX YY ZZ SS \n CL2 3_b AA File2: AA string1 AA string2 3_b string3 My ...
8
votes
1answer
108k views

Create a struct to store student data and perform statistical analysis on data

I had to create a program that had a struct that held data inside of it. I had to read an input file and take the contents and read it into an array of structs. Then, I had to solve for some ...
8
votes
3answers
812 views

Soundcloud to CSV File

This program connects to Soundcloud and turns the supplied username into a CSV file with the user's like list. The code accomplishes this although it looks rather messy and was wondering what I could ...
8
votes
3answers
378 views

Calculating effective shop inventories from CSV files

Program purpose The purpose for the program is to create a file containing the effective inventory for each stocked item stored in a collection of shops. Input and algorithm details For clarity, ...
8
votes
1answer
620 views

Parsing a CSV file with a very specific format

I'm not very good with templates, so any general tips would be appreciated. Basically this class parses a CSV file with a very specific format. My original idea for this was that I wanted this to ...
8
votes
1answer
196 views

Extract data from a DBMS, create a CSV dump and transfer it to an FTP server

First of all, my code does work as expected at least to some extent. The intention of the program is to extract data from a DBMS in order to create a CSV dump and transfer it to an FTP server. My two ...
8
votes
2answers
56 views

Streaming scheduled TV shows according to a CSV playlist

I'm very new to Python, and I've made a small program/script that will allow me to stream specific files at a given time. My idea is that I have a .csv file for each day, with one row for each movie/...
8
votes
1answer
763 views

RFC 4180-compliant CSV parser in Java

While looking at a recent question, I realized that there is no CSV parser in the standard Java library. I decided to write one that complies with RFC 4180. Highlights of the standard include: ...
7
votes
3answers
4k views

Compute mean, variance and standard deviation of CSV number file

I've written my first C++ application this week and I wanted to see if there's anything that aren't up to standards or better ways to do what I'm currently doing. For the goal of the exercise, let's ...
7
votes
3answers
255 views

Convert binary raster file to text CSV

I have a binary file input.hgt from the Shuttle Radar Topography Mission. It contains a 3601✕3601 matrix, stored as 2-Byte big-endian integer numbers. My code ...
7
votes
5answers
791 views

Finding items of related dependencies in Factorio

Factorio is a game about building factories and about automation. Items can be crafted (just like Minecraft) yourself, or crafted automatically by assembling machines. To help me expand my factories I ...
7
votes
3answers
3k views

Parsing pipe delimited lines

I am parsing the following line of code via a specific format. Line is: S|111111|87654321|Bar UK|BCreace UK|GBP|24/08/2010| The Format is: ...