Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

5
votes
1answer
35 views

Personal School Course Scheduler

A good friend of mine had the challenge of trying to build a schedule using all available times and classes through a spreadsheet... by hand. He asked me if I could build a program to generate valid ...
3
votes
1answer
32 views

Get owner of ~5 million files and folders using Powershell and Robocopy

I've written a Powershell script that uses Robocopy to get the full path of all files on a drive and then run it through a foreach loop to get the owner for each file and generate it in a CSV. The ...
3
votes
0answers
29 views

Fastest way to write large CSV file in python

I'm fairly new to python and pandas but trying to get better with it for parsing and processing large data files. I'm currently working on a project that requires me to parse a a few hundred CSV CAN ...
-4
votes
0answers
69 views

Format data for a CSV item

This function formats the text in each cell of a CSV. The goal of the function is to make sure that the sequence of characters in text will never break the cell's ...
4
votes
2answers
140 views

Iterate over a list of list names as file names

Interrogating a web API using query-url's, for each query I can get either zero hits, one hit, or multiple hits. Each of those categories needs to go into a separate CSV file for manual review or ...
6
votes
2answers
74 views

Split big file into separate files depending on column's distinct values

I have a big file, which contains column names in the first row. The data is about 20GB and consists of 10s of millions of rows. Every other row contains data which I would like to append to other ...
8
votes
2answers
45 views

Streaming scheduled TV shows according to a CSV playlist

I'm very new to Python, and I've made a small program/script that will allow me to stream specific files at a given time. My idea is that I have a .csv file for each day, with one row for each movie/...
1
vote
0answers
26 views

Grouped by First Two Digits of Account_ID of ~240 million rows of Pandas DF

I have 40 CSV files, each containing ~6 million rows, and 2 columns. The columns are account_id and account_balance. The ...
7
votes
3answers
92 views

Convert binary raster file to text CSV

I have a binary file input.hgt from the Shuttle Radar Topography Mission. It contains a 3601✕3601 matrix, stored as 2-Byte big-endian integer numbers. My code ...
1
vote
1answer
43 views

Handling periods ('.') in CSV column names with Pandas

I'm reading and processing a fairly large csv using Pandas and Python 3.7. Header names in the CSV have periods in them ('full stops', Britons say). That's a problem when you want to address data ...
4
votes
2answers
80 views

Parsing contents of a large zip file into a html parser into a .csv file

I have some zip files somewhere in the order of 2GB+ containing only html files. Each zip contains about 170,000 html files each. My code reads the file without extracting them, Passes the ...
11
votes
3answers
736 views

Aggregate prescriptions per drug in a CSV file

I have a question that is similar in spirit to this previously asked question. Nonetheless, I can't seem to figure out a suitable solution. Input: I have CSV data that looks like (FICTIONAL ...
1
vote
1answer
58 views

Scraping content from yellowpage and write them to a csv file

I've written a script in python to scrape names and phone numbers of a certain search from yellowpage traversing multiple pages. The scraper has the ability to collect all the links connected to all ...
1
vote
2answers
65 views

Powershell function to search CSV logs for certain regexes

I have a parser that goes through two different logs, both .csv files, and checks for certain lines based off the regex code that I have chosen. This parser works fine, it just takes about a minute ...
2
votes
1answer
37 views

Convenience function for writing files in CSV, JSON, etc

The McCabe code complexity of the following function is 19 as found by pytest --mccabe test.py (from my toy-project mpu). While ...
3
votes
1answer
33 views

Initializing an Android activity by reading a CSV file with image IDs, names, and descriptions

I am using Java to create an Android app. I have the main activity read a CSV file, create objects, and then save them into a singleton class so the rest of the activities in the app can access them ...
5
votes
3answers
156 views

Writing to a csv file in a customized way using scrapy

I've written a script in scrapy to grab different names and links from different pages of a website and write those parsed items ...
0
votes
2answers
55 views

Pull From Oracle into CSV File

I have written this code in C# to connect to an Oracle Database, omitting the tnsnames.ora file, then place results from a query into a CSV File. Is there more efficient way to do this? (Such as ...
8
votes
3answers
222 views

Calculating effective shop inventories from CSV files

Program purpose The purpose for the program is to create a file containing the effective inventory for each stocked item stored in a collection of shops. Input and algorithm details For clarity, ...
5
votes
1answer
106 views

Convert IEnumerable<Reads> to CSV

I have an IEnumerable of a view model that I need to convert to CSV and send back as a file response. I have written it out quite manually and I'm just seeking code ...
4
votes
2answers
576 views

Exporting a built data set to a CSV

I wrote a program to scan files, extract text, build a data set and export it to a CSV. My initial program was ugly, but it worked. I have since refactored it, but the latest version seems to run a ...
6
votes
2answers
156 views

Program to calculate some stats from Facebook messages

I've written a program that parses JSON downloaded from Facebook to calculate some stats from a conversation between two parties. It attemps to count these things: Messages grouped by day of week ...
3
votes
1answer
78 views

CSV Reader from scratch

I am new to Java and not quite familiar with its design patterns. I have tried to implement a CSV-File reader from scratch. The CSVFile constructor accepts the path to the file and a class which ...
2
votes
2answers
55 views

Parse and sort domain names and their prices

I set out to write a small Python script to tell me cheapest domains. I manually scraped for data from iwantmyname.com into a plain text file as follows: ...
0
votes
1answer
611 views

Multiply two columns of Census data and groupby

I have census data that looks like this ...
3
votes
1answer
43 views

Code runs Speed Test CLI using the python wrapper and then stores values within CSV

I wanted to make use of the speedtest-cli while also incorporating pandas into the project. My initial goal, which I am posting here, was to take the values of the speed test and incorporate them into ...
2
votes
0answers
29 views

Determine file provider based on column headers, replace columns and process data

Each day I receive files from different customers relaying the same information, but the customers use proprietary formats. For example, ...
5
votes
2answers
249 views

String Similarity using fuzzywuzzy on big data

I have a file in which I was to check the string similarity within the names in a particular column. I use fuzzywuzzy token sort ratio algorithm as it is required for my use case. here is the code, is ...
3
votes
2answers
71 views

Replacing values in a list

This is my first Powershell script ever so bear with me. I'm trying to replace values in a sharepoint list from a CSV file: ...
5
votes
1answer
60 views

Search for subsets within a 250k line text dataset

The code below works pretty well - it takes about 4 seconds (this seems too slow for me) to find a group that's located in the tail area of the text file I'm searching. The text file has ≃ 250,000 ...
0
votes
2answers
115 views

Removing entries with duplicate columns from large .vcf files

First of all, please note that the input (.vcf) file is very large (>60 GB). So I am reading the .vcf files line by line, but since I have to check for duplicate of two columns present in the input ...
8
votes
3answers
197 views

Parsing Excel file into CSV file

EDIT: Thanks everyone for your answers, this is moving quickly now and it's always essential to get better with design and best practice! I am loading in an excel file to a dataset using ...
0
votes
1answer
447 views

Generate a CSV file from a multidimensional array in php

I can tell by looking at this code that I've gone about this in the wrong way. Provided the string is correctly formatted (each index ofCSVData[][], would the ...
4
votes
4answers
111 views

Export dataset to CSV in C

I've just learnt the basics of C and I've built a working program that reads the dataset provided by my lecturer and exports it out to a CSV. However, he said that this could've been simplified (a ...
4
votes
1answer
65 views

Munging select rows in CSV files using Pandas

I'm learning Pandas. I have a project where I need to munge some CSV files and resave them as CSV. I can use dictionaries and CSV module, but decided to use DataFrames to get more exposure and ...
0
votes
0answers
42 views

Writing a table of national park data

I'm working to produce a Lookup table using the Arcpy module that contains descriptive statistics for US National Parks. The original data, found in tbl, has 12 ...
5
votes
2answers
66 views

Transpose tab-delimited table in C

I have looked online for transposing programs, but have found that they have extreme RAM use for large tables. The RAM use is so extreme I can't run most jobs on my laptop. I also have a Perl ...
3
votes
5answers
867 views

Code to initialize a .NET object from a CSV file

I am initializing the following object from a CSV file: ...
6
votes
1answer
128 views

Locating contact information from a CSV spreadsheet

I'm trying to find a more effective way to access variables within for loops. I felt that my attempt which works is not an effective way to access variables that ...
3
votes
0answers
45 views

Drupal 8 Content Import via CSV Implementation

I am implementing a way to import content into Drupal 8 via a CSV. This CSV contains various fields such as node ID, title, body, etc. It has been working great but I am very concerned about how long ...
2
votes
1answer
43 views

Consolidate multiple very large medical claim response text files for database load

Each month we receive 30 text files containing various medical claim responses. We sort these into three subfolders and then consolidate and load them into a database table for analysis. This may not ...
4
votes
1answer
39 views

Parsing positive integers from CSV file Part-2

Part-1 here This is a small CSV parser for positive integers mostly intended for personal projects. I (hopefully) improved the code from Part-1 by condensing some parts and improving others while ...
4
votes
0answers
249 views

CSV Data Table Extension

This is my library I wrote from scratch for the .net core that imports and exports CSVs from DataTable objects. I've seen a lot of huge complicated libraries that handle this and a million other ...
4
votes
1answer
56 views

Parsing positive integers from CSV file

I Needed to parse some positive integers with the following constraints: If possible use C++ only (no C API) Needs to be able to process parsed data after each newline Should be reasonably fast ...
1
vote
1answer
48 views

Read integers from text file and write to CSV file

Background: I've recently started learning bash due to my new job in a VFX company. We backup all of our media to LTO tapes (one master and one clone). I was tasked with writing a script that split ...
2
votes
2answers
173 views

ASCII encryption program in Python

I have recently written a program that reads information from a CSV file and puts it into lists. An encryption sub procedure converts the characters to ASCII, adds 2 then converts them back to ...
3
votes
0answers
62 views

Filtering OTC market data

It's been one month I do programming in python. Hence, my experience is a bit limited. Here is what I've done so far. The program is only filtering OTC market data according to certain features. Could ...
0
votes
1answer
459 views

Convert Excel files to CSV

I wrote the following script to run through several excel files and format them before saving as a CSV for upload to a Quickbase app I'm creating. Each excel file is roughly 100k lines and the code ...
7
votes
2answers
244 views

Parse one table out of CSV with multiple titled tables with the Python CSV module

I'm looking to parse CSV files containing multiple tables using Python 3's csv module. These complex CSVs are not unlike the toy example below. My goal is to make ...
2
votes
1answer
69 views

Crap (from the Past) Scraper

I listen to a podcast called Crap from the Past. It's a radio show that airs weekly in Minneapolis and specializes in playing music from my childhood. The show has been on the air for ~26 years. I ...