A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

learn more… | top users | synonyms

1
vote
0answers
8 views

Efficiently filter a large (100gb+) csv file (v3)

I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this question. The problem is as follows: I need to filter ...
4
votes
2answers
95 views

Efficiently filter a large (100gb+) csv file (v2)

EDIT: This question is followed up by this question. I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this ...
2
votes
1answer
35 views

Populate a .txt document with a CSV spreadsheet using multiple templates, outputting documents for each data row

I'm looking for feedback on functionality, PEP8, advice, etc as a learning tool. I'm not sure how to tell if this program should be broken into separate functions or combined as one. While hand ...
3
votes
1answer
69 views

Create a struct to store student data and perform statistical analysis on data

I had to create a program that had a struct that held data inside of it. I had to read an input file and take the contents and read it into an array of structs. Then, I had to solve for some ...
3
votes
3answers
51 views

Merging three CSV files with a primary key

This is my first time coding in my life. I don't really know much about coding. I googled whatever I need and combined them all together. It works great, but want to know if there is any improvement ...
2
votes
1answer
38 views

Tweeting statistics on shared links

I tend to be rather prolific in my sharing of things I find online. I'm interested in tracking my own habits to make them more effective. Therefore, I submit the above to be critiqued by the lot of ...
0
votes
0answers
13 views

Converts large csv dataset to hash then imports into MySQL/SQLITE3 database using Sequel

The main line in this script that allows Sequel to insert records into the database table is ...
1
vote
1answer
53 views

Reading two csv files as dictionaries with differing key definitions

My script reads two .csv files and generates one dictionary per .csv file. ...
4
votes
1answer
106 views

Printing lines in a file, filtering by the first column

I am setting up a project which involves a lot of data analysis, each data point represented by a line in a large text file. In order to test for certain types of data, I added a tag to each line (It ...
1
vote
1answer
78 views

Reduce columns to produce product keycodes

I have written the following code that achieves my desired goal of converting a "raw" CSV file into a keycoded list. The idea is that a person will get the first product they ask for, and English ...
2
votes
2answers
63 views

Parsing a single CSV line into a list of strings

I've written this method to replace an older method that was much simpler, but used the regex split method and couldn't tell if a comma was in quotes/brackets/etc. and didn't read double quotes as ...
2
votes
1answer
77 views

Efficiently filter a large (100gb+) csv file (v1)

EDIT: This question was followed up by this question, with improved code. I have a set of pretty large files (>2 GB, over 30m rows) containing intraday data in the following format: ...
3
votes
0answers
34 views

Utility that decodes and logs UDP packets

I have written the following utility, as my first non-tutorial program in Go. The purpose of the utility is to connect to a torque/force sensor (aka load-cell) via UDP; to send an initialization ...
1
vote
0answers
81 views

Using BeautifulSoup to scrape various tables and combine in a csv file

Looking for some feedback/criticism/improvements to a piece of code I've written. It works as intended but I'd like some feedback! Quick overview of what it does. A page contains a table of links, ...
-1
votes
2answers
88 views

Parsing a big text file, extract data & store it in a CSV file

I have a big log file (say 1-3 Gb) which I need to parse, extract data & save it in a CSV file. Text File Data ...
3
votes
4answers
229 views

Splitting an array at commas

I was given a file of size 46KB full of names (around ~5000), in the format as shown below: "MARY","PATRICIA","LINDA","BARBARA","ELIZABETH"..."DARRELL","ALONSO" ...
5
votes
3answers
63 views
0
votes
0answers
20 views

Jekyll plugin for creating CSV files

I recently created a gem for Jekyll plugin to create some CSV files. I put a little more effort into it and tried to make it something that others might be able to use. I'm a little lost on how to ...
1
vote
1answer
59 views

Corrects the encoding of a CSV file in Qt

I'm hoping for a little bit of a code review to ensure I am doing UTF-8 right. I am developing a "csv_fixer" that allows a user to select the encoding of the file so we can correctly handle encoding ...
3
votes
1answer
54 views

Python CSV command line tool to import CSV into MySQL

I've written my first Python program longer than 15 rows and, of course, it looks terrible. I'd be very glad to receive comments and suggestion about how to implement a similar tool correctly. ...
0
votes
2answers
57 views

Transferring a CSV file to a database while merging some columns

Say I am given a csv file that I want to import into an sqlite3 database. The first two columns in the csv file contain unique, important information while the rest of the columns are really a list ...
6
votes
2answers
165 views

Extracting keywords from 3 billion CSV records

I am processing around 3 billion records using this piece of code. It's pretty slow; it would be really helpful if you could suggest better ways to do this. Update #1 Based on Nizam Mohammed ...
3
votes
1answer
82 views

Combining CSVs using ruby to match headers

I am trying to write a ruby script that will take all of the CSVs in a given directory and map their headers out so that duplicate columns will match up and columns unique to a file will have their ...
5
votes
1answer
47 views

Interface to a CSV dinosaur database

I have just finished with the first Ruby exercise on Level Up Rails, and wanted to get an idea on how I can refactor the code. You can find the original exercise on github - including the ...
7
votes
1answer
135 views

Parsing > 30 GB file in least amount of time

My Goal: To efficiently move data from Amazon S3 to Amazon Redshift. Basically, I am moving all CSV files on my S3 to Redshift using the below code. I parse through part of the file, build a table ...
8
votes
4answers
479 views

Reading a text file that contains several CSV-like tables

I wrote some extension methods to read CSV-styled text directly into a datatable or dataset or write them to this format. Is it right to use the methods as extensions or should I create a separate ...
0
votes
0answers
32 views

Stream file to opencsv with the least amount of I/O possible

I have the following code snippet where I'm picking up a file from an ftp server and then streaming it to opencsv. I have very little experience with streams, so I'd like to have someone review this ...
2
votes
1answer
40 views

CSV manipulation with Python

I've prepared a script to manipulate csv files in Windows that simply adds quotes as requested. The script creates a backup of the original files and overwrite the wrong originals in its same folder. ...
8
votes
3answers
367 views

Soundcloud to CSV File

This program connects to Soundcloud and turns the supplied username into a CSV file with the user's like list. The code accomplishes this although it looks rather messy and was wondering what I could ...
4
votes
2answers
68 views

Tic Tac Toe Game Sequence Generator

I want my code to write to a .csv file a bunch of patterns for Tic Tac Toe games (30000). I want to have 9 columns numbered 1 through 9 and in each have the number of the space in the grid that was ...
6
votes
3answers
285 views

Manipulating CSV files to regular text file

At my job, often I have to manually convert *.csv Files structured like: number,number\n number,number\n number,number\n ... To a regular *.txt file with the ...
3
votes
1answer
117 views

Reading from a CSV file in C++

I wrote this code to read from a CSV text file (containing data such as 12,3,568,48,3,8 with no more than 3 digits to each number). It stores the numbers as char arrays in the vector ...
4
votes
1answer
80 views

Script to generate documents based on conditionals and CSV data

I wrote this script a while back to generate press releases. It consumes a csv of data where each row contains the data for a country and fills in values in pre-written text based on the data. In its ...
4
votes
5answers
147 views

Optimising single-delimiter string tokenisation

I am trying to optimise my tokenizing of tab delimited strings: ...
0
votes
1answer
124 views

Need fast csv parser for Python to parse 80GB csv file [closed]

I am looking for the fastest way to parse an 80GB csv file with 300 columns in Python. The csv file does not have backquote commas, e.g a,"blah,blah,blah",c I ...
3
votes
0answers
22 views

Identifying surface events happening at specific time intervals

Here is some code I wrote to surface Mint.com transactions that occur at monthly intervals, in order to identify subscriptions I may be paying for without realizing it. I'd like to have some friends ...
16
votes
2answers
259 views

API for looking up Japanese postal code

Most of my experience is in Python, Java and embedded C, and I need to learn Scala and Play for a new job. As an exercise, I decided to create a very simple API for looking up area by postal code, ...
3
votes
3answers
331 views

Inserting persons in file into SQLite

I've just started working with SQLite in C# to test various features of an application. In building my first SQLite example I wanted to insert a large .csv into a table (Person) with two columns, A ...
2
votes
1answer
231 views

Optimizing for data import in Neo4j using py2neo

Here is my code for importing from a .csv to a neo4j graph using py2neo and cypher statements. I've noticed that it slows down significantly the bigger the graph gets. It takes several seconds just to ...
1
vote
1answer
118 views

Transposing rows and columns for file contents [closed]

This program transposes rows and columns for file contents: ...
0
votes
1answer
100 views

Convert Array CSV to Object using Header

Takes the first array in the array and uses that to set the properties of an array-object. ...
3
votes
1answer
89 views

Confidence score calculation

I was looking for a way to calculate a score 'x' for each pair of elements in two separate arrays. The goal of the code is to return an output array containing the score for each entry in the input ...
3
votes
2answers
103 views

Creating large CSV file

I have a performance problem when trying to create a csv file starting from another csv file. This is how the original file looks: ...
2
votes
1answer
32 views

Split CSV by Repeated cells python

Any time that a row ID (oddly placed in column 8, i.e. row[7]) is repeated after the first instance, I want to write those rows into a second file. The code I'm using is extremely slow -- it's a ...
2
votes
2answers
87 views

CSV File Parser in Ruby - Attempt No. 2

The following is my second attempt at a basic CSV parser in Ruby. I've addressed suggestions that I've got for my previous version here. I have omitted all test cases - the code listing would be too ...
5
votes
3answers
60 views

Some simple Perl and Regex

Reads from a flat file of three columns, delimited by at least one tab character. Filters the three columns based on input; returns list of corresponding emails. Arguments passed like ...
2
votes
1answer
145 views

Parse a csv file and create a dictionary of partial results

I have a bunch of .csv files which I have to read and look for data. The .csv file is of the format: ...
2
votes
0answers
31 views

Parsing CSVs for bulk database insertions

I'm writing this as (eventually) part of a larger program. This will serve as the bulk data insertion from .csv files we parse. I'm primarily looking for parts that are breaking Go's formatting, ...
4
votes
1answer
249 views

Combat log parser for World of Warcraft

This is part of a bigger project I'm doing to get better at JavaScript. It's basically the beginnings of a combat log parser for World of Warcraft. It looks a bit messy right now and I could really ...
4
votes
3answers
1k views

Splitting and printing comma-separated values

I stumbled upon a question on SO asking how to split a comma-separated string into individual values. Since it's been a while since I've had any good reason to write C I'd like to ask for some ...