A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

learn more… | top users | synonyms

15
votes
5answers
1k views

Inventorizing the universe

I recently started re-factoring a project I hadn't touched in many months. The original purpose was to be an inventory management system for a game I played, Ogame. In this game you play inside a ...
3
votes
2answers
109 views

Converting CSV files to Excel workbooks

My converter converts all CSV files in the subfolders of folders 1, 2 & 3 into Excel workbooks. As of now, I am converting using codes for each folders. I previously tried to combine those into ...
3
votes
1answer
68 views

Converting a CSV file to a social network graph for Pajek

The code will ask for a directory and will then read the CSV file and then generate a file in that directory describing a social network for Pajek. Does anyone know a way to load this CSV file to a ...
1
vote
0answers
18 views

Filter CSV file and display it using Highcharts

I created a function which reads CSV file and displays those values in a graphical form using Highcharts. Although it works, I have to read the CSV file twice because first: I need to get all the ...
3
votes
0answers
31 views

Extracting data from a CSV file on the internet [duplicate]

The code extracts data from a CSV file stored on the web. My sheet, 'New Data', stores 2 days' worth of data so that when I run it next time, it pulls the data from the web, compares to the data I ...
6
votes
3answers
104 views

Extracting data from CSV file on the internet

This is my code I made that extracts data from a .csv file stored on the web. My sheet called 'New Data' stores this data for 2 days. I store 2 days worth of data so that when I run it next time, it ...
2
votes
0answers
21 views

Consolidating a raw csv of two ids to grouped set of ids in json

The file I'm processing here is an outputted file from a Java library called duke. Its goal was go through all purchases made on a site and group them into a single customer. The output is a csv ...
3
votes
1answer
64 views

Rating tennis players in a database, taking days to run

I have this project in data analysis for creating a ranking of tennis players. Currently, it takes more than 6 days to run on my computer. Can you review the code and see where's the problem? ...
2
votes
2answers
44 views

Python script to read CSVs and regroup it into the data types I need

This code does the following: Read CSV file and determine sub_type based on series_desc. Turn row into dictionaries ...
0
votes
2answers
25 views

Comparing two columns in two different rows

I want to go through each line of the a .csv file and compare to see if the first field of line 1 is the same as first field of next line and so on. If it finds a match then I would like to ignore ...
5
votes
1answer
156 views

Extracting emails from a file and writing them to another file

The code below works fine for me. Is there any way that I can improve this code more? Note: delimiter is single whitespace only for email address. getmail.py ...
-2
votes
0answers
40 views

Cleansing data using Python

Please suggest a faster way of cleansing data. This implementation was ran on 100000 records. ...
5
votes
1answer
30 views

Import Excel, unpivot data, output tab-delimited for SQL Import

I have a client GL file (Excel) that needs to be regularly imported to SQL Server (as tab-delimited txt). Sample (simplified): ...
2
votes
1answer
47 views

Read and output CSV content OO PHP

Okay, I am wanting to expand my armoury within PHP, so I've been researching OO PHP. With my researched knowledge I went on to create a quick script that reads a CSV file and outputs the results. CSV ...
4
votes
2answers
79 views

CSV import-export in Python

I have finished a really short and small import / export script using Python 2.7, and now I would like to structure it using classes and methods where possible. Could anybody give me some advice on ...
3
votes
3answers
166 views

Parsing multiple line records using Java 8 Streams

There was a suggestion in this StackOverflow question to move this question here. I added some more information and restructured the question a bit. I'm trying to parse the following file which ...
1
vote
1answer
49 views

Finite state machine for CSV data

I want to read a file containing comma-separated values, so have written a finite state machine: ...
2
votes
1answer
39 views

Denormalizing an OrderedDict to CSV

Cross post from stackoverflow, as suggested. I have an ordereddict, and I need to store it in CSV file. Input: ...
7
votes
1answer
109 views

Creating a Wrapper for CSV Data

I am trying to complete a Ruby coding exercise. The specifics of the excercise can be found on a the level_up_exercises GitHub repo. This repo also contains the CSV file used in the exercise. I ...
0
votes
1answer
73 views

Split excel file with multiple sheets, manipulate the data and create final out file

I have an excel file with 20+ separate sheets containing tables of data. My script iterates through each sheet, manipulates the data into the format I want it and then saves it to a final output file. ...
3
votes
1answer
49 views

Parsing large CSV in Perl

I am getting Out of Memory errors when I try to parse through a large CSV file (2.5Gb). My computer has 32Gb of Memory but Perl uses all of it up. The CSV has 2 ...
1
vote
0answers
67 views

Using BeautifulSoup to scrape various tables and combine in a .csv file

A page contains a table of links, each link contains a table relevant to the link (a subject). Create a list of these links to pass to the function called ...
1
vote
0answers
37 views

Ruby Script to modify and split CSVs

I have written the following script that is pointed at a CSV through it's file name and then splits the file into "drops" (for mailings) and does a couple of operations on it. As of now it works, but ...
4
votes
1answer
113 views

Efficiently filter a large (100gb+) csv file (v3)

I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this question. The problem is as follows: I need to filter ...
4
votes
2answers
123 views

Efficiently filter a large (100gb+) csv file (v2)

EDIT: This question is followed up by this question. I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this ...
2
votes
1answer
77 views

Populate a .txt document with a CSV spreadsheet using multiple templates, outputting documents for each data row

I'm looking for feedback on functionality, PEP8, advice, etc as a learning tool. I'm not sure how to tell if this program should be broken into separate functions or combined as one. While hand ...
3
votes
1answer
143 views

Create a struct to store student data and perform statistical analysis on data

I had to create a program that had a struct that held data inside of it. I had to read an input file and take the contents and read it into an array of structs. Then, I had to solve for some ...
3
votes
3answers
75 views

Merging three CSV files with a primary key

This is my first time coding in my life. I don't really know much about coding. I googled whatever I need and combined them all together. It works great, but want to know if there is any improvement ...
2
votes
1answer
45 views

Tweeting statistics on shared links

I tend to be rather prolific in my sharing of things I find online. I'm interested in tracking my own habits to make them more effective. Therefore, I submit the above to be critiqued by the lot of ...
1
vote
0answers
47 views

Converts large csv dataset to hash then imports into MySQL/SQLITE3 database using Sequel [closed]

The main line in this script that allows Sequel to insert records into the database table is: ...
1
vote
1answer
61 views

Reading two csv files as dictionaries with differing key definitions

My script reads two .csv files and generates one dictionary per .csv file. ...
4
votes
1answer
122 views

Printing lines in a file, filtering by the first column

I am setting up a project which involves a lot of data analysis, each data point represented by a line in a large text file. In order to test for certain types of data, I added a tag to each line (It ...
1
vote
1answer
87 views

Reduce columns to produce product keycodes

I have written the following code that achieves my desired goal of converting a "raw" CSV file into a keycoded list. The idea is that a person will get the first product they ask for, and English ...
3
votes
3answers
92 views

Parsing a single CSV line into a list of strings

I've written this method to replace an older method that was much simpler, but used the regex split method and couldn't tell if a comma was in quotes/brackets/etc. and didn't read double quotes as ...
2
votes
1answer
99 views

Efficiently filter a large (100gb+) csv file (v1)

EDIT: This question was followed up by this question, with improved code. I have a set of pretty large files (>2 GB, over 30m rows) containing intraday data in the following format: ...
3
votes
0answers
41 views

Utility that decodes and logs UDP packets

I have written the following utility, as my first non-tutorial program in Go. The purpose of the utility is to connect to a torque/force sensor (aka load-cell) via UDP; to send an initialization ...
-1
votes
2answers
183 views

Parsing a big text file, extract data & store it in a CSV file

I have a big log file (say 1-3 Gb) which I need to parse, extract data & save it in a CSV file. Text File Data ...
3
votes
4answers
239 views

Splitting an array at commas

I was given a file of size 46KB full of names (around ~5000), in the format as shown below: "MARY","PATRICIA","LINDA","BARBARA","ELIZABETH"..."DARRELL","ALONSO" ...
5
votes
3answers
63 views
0
votes
0answers
32 views

Jekyll plugin for creating CSV files

I recently created a gem for Jekyll plugin to create some CSV files. I put a little more effort into it and tried to make it something that others might be able to use. I'm a little lost on how to ...
1
vote
1answer
93 views

Corrects the encoding of a CSV file in Qt

I'm hoping for a little bit of a code review to ensure I am doing UTF-8 right. I am developing a "csv_fixer" that allows a user to select the encoding of the file so we can correctly handle encoding ...
3
votes
1answer
119 views

Python CSV command line tool to import CSV into MySQL

I've written my first Python program longer than 15 rows and, of course, it looks terrible. I'd be very glad to receive comments and suggestion about how to implement a similar tool correctly. ...
0
votes
2answers
62 views

Transferring a CSV file to a database while merging some columns

Say I am given a csv file that I want to import into an sqlite3 database. The first two columns in the csv file contain unique, important information while the rest of the columns are really a list ...
6
votes
2answers
190 views

Extracting keywords from 3 billion CSV records

I am processing around 3 billion records using this piece of code. It's pretty slow; it would be really helpful if you could suggest better ways to do this. Update #1 Based on Nizam Mohammed ...
3
votes
1answer
180 views

Combining CSVs using ruby to match headers

I am trying to write a ruby script that will take all of the CSVs in a given directory and map their headers out so that duplicate columns will match up and columns unique to a file will have their ...
6
votes
1answer
72 views

Interface to a CSV dinosaur database

I have just finished with the first Ruby exercise on Level Up Rails, and wanted to get an idea on how I can refactor the code. You can find the original exercise on github - including the ...
7
votes
1answer
159 views

Parsing > 30 GB file in least amount of time

My Goal: To efficiently move data from Amazon S3 to Amazon Redshift. Basically, I am moving all CSV files on my S3 to Redshift using the below code. I parse through part of the file, build a table ...
8
votes
4answers
781 views

Reading a text file that contains several CSV-like tables

I wrote some extension methods to read CSV-styled text directly into a datatable or dataset or write them to this format. Is it right to use the methods as extensions or should I create a separate ...
1
vote
1answer
77 views

Stream file to OpenCSV with the least amount of I/O possible

I have the following code snippet where I'm picking up a file from an FTP server and then streaming it to OpenCSV. I have very little experience with streams, so I'd like to have someone review this ...
2
votes
1answer
60 views

CSV manipulation with Python

I've prepared a script to manipulate csv files in Windows that simply adds quotes as requested. The script creates a backup of the original files and overwrite the wrong originals in its same folder. ...