Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
39
votes
1answer
1k views

Utility that decodes and logs UDP packets

I have written the following utility, as my first non-tutorial program in Go. The purpose of the utility is to connect to a torque/force sensor (aka load-cell) via UDP; to send an initialization ...
32
votes
6answers
6k views

Counting rows in a CSV file that correspond to a database row, each with a million records

I have two DataTables: dt: is populated from a CSV file with over 1.7 million rows dataStructure.Tables["AccountData"]: is ...
30
votes
6answers
3k views

Python web-scraper to download table of transistor counts from Wikipedia

I have been looking for answers for how to easily scrape data from Wikipedia into a CSV file with Beautiful Soup. This is the code so far. Is there an easier way to do it? ...
25
votes
5answers
2k views

Inventorizing the universe

I recently started re-factoring a project I hadn't touched in many months. The original purpose was to be an inventory management system for a game I played, Ogame. In this game you play inside a ...
23
votes
5answers
21k views

Parsing CSV to specific format

I've written this code to read CSV files written in a specific format. I would like to gather some feedback on where it could be improved. I'm trying to get into the Test-Driven-Development ideology ...
23
votes
5answers
2k views

Generating CSV strings for various 3rd party utilities

I'm generating CSV strings for various 3rd party utilities and this section of code gets repeated in many classes. Is there a better way to generate this string? ...
22
votes
4answers
8k views

Regex to parse semicolon-delimited fields is too slow

I have a file with just 3500 lines like these: filecontent= "13P397;Fotostuff;t;IBM;IBM lalala 123|IBM lalala 1234;28.000 things;;IBMlalala123|IBMlalala1234" ...
21
votes
4answers
7k views

System that manages employee data for managers

This code is for an employee management system. The goal that I am trying to reach is to allow managers to access data of their employees more easily. The data that is entered is saved to a CSV file ...
19
votes
4answers
8k views

Web scraping the titles and descriptions of trending YouTube videos

This scrapes the titles and descriptions of trending YouTube videos and writes them to a CSV file. What improvements can I make? ...
19
votes
8answers
50k views

Java function to read a CSV file

I have a java function that reads a csv and returns its content as a Vector<Vector<String>>. The function seems to work as I need it, but my gut ...
19
votes
2answers
2k views

API for looking up Japanese postal code

Most of my experience is in Python, Java and embedded C, and I need to learn Scala and Play for a new job. As an exercise, I decided to create a very simple API for looking up area by postal code, ...
17
votes
4answers
1k views

Sensor logger for Raspberry Pi in a stratospheric probe

I'm writing a Python Script for a Raspberry Pi to measure different sensors. We are planning to send the Pi with that Script running to the stratosphere, so the power usage for the Pi is limited. I ...
16
votes
1answer
36k views

Efficiently filter a large (100gb+) csv file (v3)

I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this question. The problem is as follows: I need to filter ...
12
votes
3answers
1k views

Retrieving a csv header

I have two seperate calls that returns a .csv file. I need to extract a couple of headers from the files depending on which one is called. The headers does not always sit in the same index, and the <...
12
votes
2answers
24k views

CSV reader using StreamReader and LINQ

Mostly due to readability, I am using the following code to find a specific csv file on a drive read that CSV file using a streamReader parse it into a ...
12
votes
1answer
868 views

Find the peak stock price for each company from CSV data

During the hiring process, a company gave me this test: Q) Consider Share prices for a N number of companies given for each month since year 1990 in a CSV file. Format of the file is as below with ...
11
votes
4answers
2k views

Country code lookup for each line in a CSV file

I'm trying to write a program that has a for loop, inside of which I have lambda expression to get a certain value. The loop has about 20,000 iterations. Including ...
11
votes
4answers
1k views

Parsing RFC 4180 CSV with GOTOs

One of my data-import tools needs to support CSV files. I thought that parsing CSV is such a simple task that I didn't want to use any any external libraries for that. So here is one more RFC 4180 CSV ...
11
votes
3answers
961 views

Aggregate prescriptions per drug in a CSV file

I have a question that is similar in spirit to this previously asked question. Nonetheless, I can't seem to figure out a suitable solution. Input: I have CSV data that looks like (FICTIONAL ...
11
votes
3answers
9k views

Convert all CSV files in a given directory to JSON using Python

I am attempting to convert all files with the csv extension in a given directory to json with this python script. I am wondering if there is a better and more efficient way to do this? Here is my ...
11
votes
2answers
439 views

Averaging numbers from a file, obeying Single Responsibility Principle

Task: Given comma_separated.txt file with string: 1,2,3,4,5,6,7,8,9,10,11,12...,n Calculate the average ...
11
votes
3answers
438 views

Print Student Names from CSV

Description Print all student names from a CSV file. CSV Format: CSV file does not contain string delimited by double quotes. None of the cells has "," character ...
11
votes
2answers
1k views

Showing virus scan results from an API and a CSV file

I am utilizing an API module for interacting with virustotal.com in order to get AntiVirus results based of SHA256 hashes. The code I have is working but I feel like it can be improved greatly. I ...
10
votes
2answers
1k views

Reading a repetitive file with repetitive code

I made this code for take some data from a .txt file. The text file is a large list of data from temperature with a format that is repetitive and have some lines at the beginning with the station ...
10
votes
2answers
4k views

Read/write a pipe-delimited file line by line with some simple text manipulation

This code that I wrote is supposed to read/write a pipe-delimited file line by line to a new file with some simple text manipulation. (It also adds two new columns) and publishes a "Status Update" ...
10
votes
2answers
6k views

Parsing CSV file with BufferedReader vs Scanner

I have to parse a file line-by-line and in single line I have split by ",". The first string is the name and the second is the count. Finally, I have to display the Key and Count, for example: ...
10
votes
1answer
7k views

Faster Django CSV generation for several million database entries

I am writing a CSV file generator that's filtering through about seven million database entries (MySQL backend). This part is especially slow and I was wondering if there is a way to make it much ...
10
votes
1answer
53k views

Writing lists to .csv file

I'm teaching myself programming, using Python as my initial weapon of choice. I have learnt a few basics and decided to set myself the challenge of asking the user for a list of names, adding the ...
10
votes
2answers
477 views

In memory data access for CSV files that may have changed

I'm reading data from CSV files and store the result in a simple DataContext instance that gets injected throughout my application. ...
10
votes
1answer
275 views

Loading text into an array when fields are in random order

This script reads records from a CSV file of customer names and addresses. The first record of the file is the header (field names); the fields could be in any order, and sometimes have different ...
9
votes
5answers
2k views

Convert total minutes into “Hours:minute:seconds” for marathon data

The point of the script below is to read in the 10th column which is the total number of minutes a runner took to finish the marathon and convert it into form of "hours:minutes:seconds". Is there a ...
9
votes
2answers
1k views

Are there ways to speed up this string fuzzy matching in Golang?

I have a piece of python code doing fuzzy matching which works very well and is pretty fast. For reference, it uses the following files: https://raw.githubusercontent.com/datasets/s-and-p-500-...
9
votes
2answers
834 views

Searching for data from file1 in file2

In order to answer a given question on Ask Ubuntu I came up with two scripts for doing the following task: Compare two files for an occurrence of column no. 6 value in file1 occurs in file2. On ...
9
votes
3answers
233 views

Collate votes on MP3 files to list them by popularity

For my first Python project, I decided to create a script that would automatically collate MP3 files into a music countdown. I wanted to 'learn by doing', so I dived straight in and used the Python ...
9
votes
3answers
202 views

Performance - Read large amount of XMLs and load into single csv

I am dealing with a large amount of XML files which I obtained from here https://clinicaltrials.gov/ct2/resources/download#DownloadAllData. The download yields around 300.000 XML files of similar ...
9
votes
2answers
3k views

Getting lists of values from a CSV

I have a function that takes a column title, and a response.body from a urllib GET (I already know the body contains text/csv), and iterates through the data to build a list of values to be returned. ...
9
votes
1answer
111 views

Text menu to browse CSV data about seal species

I am writing in C++, I have completed a small program which parses CSV files from IUCN and presents the species data using menus. I used a lot of loops. It works, however it is buggy when processing ...
8
votes
4answers
5k views

Reading a text file that contains several CSV-like tables

I wrote some extension methods to read CSV-styled text directly into a datatable or dataset or write them to this format. Is it right to use the methods as extensions or should I create a separate ...
8
votes
4answers
11k views

Convert XML to CSV

I'm pretty sure this code can be optimized, but I'm not talented enough in Linq to do it myself. Here's what I'm trying to do: I have an XML file that needs to be converted into a .csv file. The XML ...
8
votes
3answers
4k views

Parsing Excel file into CSV file

EDIT: Thanks everyone for your answers, this is moving quickly now and it's always essential to get better with design and best practice! I am loading in an excel file to a dataset using ...
8
votes
2answers
8k views

String Similarity using fuzzywuzzy on big data

I have a file in which I was to check the string similarity within the names in a particular column. I use fuzzywuzzy token sort ratio algorithm as it is required for my use case. here is the code, is ...
8
votes
2answers
5k views

Parse one table out of CSV with multiple titled tables with the Python CSV module

I'm looking to parse CSV files containing multiple tables using Python 3's csv module. These complex CSVs are not unlike the toy example below. My goal is to make ...
8
votes
3answers
175 views

MP3 Vote Collator - Final

Here is the completed code for my first Python project, for which I received some help and feedback here: Collate votes on MP3 files to list them by popularity Seeing as there have been considerable ...
8
votes
1answer
617 views

Outputting a .txt file from a .csv file

I wrote this little script in PHP a bit ago. It takes a .csv file and outputs a .txt file with certain data arranged in a "psuedo-array" looking format. I recently started messing around with Ruby so ...
8
votes
2answers
96 views

Iterate list to map entries in Python

I have two files, namely: File1: CL1 AA XX YY ZZ SS \n CL2 3_b AA File2: AA string1 AA string2 3_b string3 My ...
8
votes
1answer
114k views

Create a struct to store student data and perform statistical analysis on data

I had to create a program that had a struct that held data inside of it. I had to read an input file and take the contents and read it into an array of structs. Then, I had to solve for some ...
8
votes
3answers
835 views

Soundcloud to CSV File

This program connects to Soundcloud and turns the supplied username into a CSV file with the user's like list. The code accomplishes this although it looks rather messy and was wondering what I could ...
8
votes
2answers
708 views

Analyzing records of London Underground outages

You have a file that describes historic data of the London Underground showing dates when stations have been unavailable. The file contains these fields, separated by a pipe character: Date: ...
8
votes
2answers
902 views

Getting current weather data using OpenWeatherMap API

I would really appreciate if someone could take a look at this code. I am not very advanced, so ANY kind of feedback and critique would be precious to me. What this program is supposed to do is to go ...
8
votes
3answers
474 views

Calculating effective shop inventories from CSV files

Program purpose The purpose for the program is to create a file containing the effective inventory for each stocked item stored in a collection of shops. Input and algorithm details For clarity, ...

1
2 3 4 5
12