Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

2
votes
1answer
82 views

Hotel booking system help on how to book nights and memberships

I am doing a GCSE NEA project for practice. Customer Loyalty Scheme The managers of Crawdale Hotel Group have decided to update their customer loyalty scheme. Customers joining the scheme ...
0
votes
1answer
58 views

Sorting CSV table in Powershell

I'm wondering if some PS gurus can advise on whether my method for sorting and recombining some data fields by using an array of PSCustomObjects is as efficient as it might be. I'm importing a CSV ...
4
votes
0answers
64 views

Python CSV to XML converter

I'm creating an application that reads in data from a CSV file and creates an XML file using LXML. The below code works as expected. However, before I develop it further I would like to refactor it ...
2
votes
1answer
49 views

Read a directory of CSV files in R

I'm writing a lot of R in my spare time and am trying to get into good habits by using TidyVerse as much as possible. A large part of this has required reading entire directories of dated log files ...
1
vote
1answer
36 views

CSV file cell slicer

This program is supposed to find and slice any cell within a CSV file which contains more than a defined number of characters. The files can be pretty large, so being a beginner I would like to know ...
2
votes
0answers
18 views

Sorting files into subdirectories based on keyword scores

This script sorts files from current directory into known subdirectories. it is built for speed because the keyword database used can be up to 20kb. each line of the database has ...
4
votes
2answers
72 views

Calculating travel statistics based on CSV data

I took a code test for acme transport company and was rejected. That's ok. I'm still learning how to code. I didn't get much feedback so that's a bit frustrating. Can anyone here provide tips on how ...
5
votes
1answer
48 views

Personal School Course Scheduler

A good friend of mine had the challenge of trying to build a schedule using all available times and classes through a spreadsheet... by hand. He asked me if I could build a program to generate valid ...
3
votes
1answer
48 views

Get owner of ~5 million files and folders using Powershell and Robocopy

I've written a Powershell script that uses Robocopy to get the full path of all files on a drive and then run it through a foreach loop to get the owner for each file and generate it in a CSV. The ...
3
votes
0answers
238 views

Fastest way to write large CSV file in python

I'm fairly new to python and pandas but trying to get better with it for parsing and processing large data files. I'm currently working on a project that requires me to parse a a few hundred CSV CAN ...
-3
votes
1answer
107 views

Format data for a CSV item

This function formats the text in each cell of a CSV. The goal of the function is to make sure that the sequence of characters in text will never break the cell's ...
4
votes
2answers
148 views

Iterate over a list of list names as file names

Interrogating a web API using query-url's, for each query I can get either zero hits, one hit, or multiple hits. Each of those categories needs to go into a separate CSV file for manual review or ...
6
votes
2answers
136 views

Split big file into separate files depending on column's distinct values

I have a big file, which contains column names in the first row. The data is about 20GB and consists of 10s of millions of rows. Every other row contains data which I would like to append to other ...
8
votes
2answers
47 views

Streaming scheduled TV shows according to a CSV playlist

I'm very new to Python, and I've made a small program/script that will allow me to stream specific files at a given time. My idea is that I have a .csv file for each day, with one row for each movie/...
1
vote
0answers
29 views

Grouped by First Two Digits of Account_ID of ~240 million rows of Pandas DF

I have 40 CSV files, each containing ~6 million rows, and 2 columns. The columns are account_id and account_balance. The ...
7
votes
3answers
98 views

Convert binary raster file to text CSV

I have a binary file input.hgt from the Shuttle Radar Topography Mission. It contains a 3601✕3601 matrix, stored as 2-Byte big-endian integer numbers. My code ...
1
vote
1answer
63 views

Handling periods ('.') in CSV column names with Pandas

I'm reading and processing a fairly large csv using Pandas and Python 3.7. Header names in the CSV have periods in them ('full stops', Britons say). That's a problem when you want to address data ...
4
votes
2answers
85 views

Parsing contents of a large zip file into a html parser into a .csv file

I have some zip files somewhere in the order of 2GB+ containing only html files. Each zip contains about 170,000 html files each. My code reads the file without extracting them, Passes the ...
11
votes
3answers
749 views

Aggregate prescriptions per drug in a CSV file

I have a question that is similar in spirit to this previously asked question. Nonetheless, I can't seem to figure out a suitable solution. Input: I have CSV data that looks like (FICTIONAL ...
1
vote
1answer
64 views

Scraping content from yellowpage and write them to a csv file

I've written a script in python to scrape names and phone numbers of a certain search from yellowpage traversing multiple pages. The scraper has the ability to collect all the links connected to all ...
1
vote
2answers
69 views

Powershell function to search CSV logs for certain regexes

I have a parser that goes through two different logs, both .csv files, and checks for certain lines based off the regex code that I have chosen. This parser works fine, it just takes about a minute ...
2
votes
1answer
44 views

Convenience function for writing files in CSV, JSON, etc

The McCabe code complexity of the following function is 19 as found by pytest --mccabe test.py (from my toy-project mpu). While ...
3
votes
1answer
46 views

Initializing an Android activity by reading a CSV file with image IDs, names, and descriptions

I am using Java to create an Android app. I have the main activity read a CSV file, create objects, and then save them into a singleton class so the rest of the activities in the app can access them ...
5
votes
3answers
519 views

Writing to a csv file in a customized way using scrapy

I've written a script in scrapy to grab different names and links from different pages of a website and write those parsed items ...
0
votes
2answers
67 views

Pull From Oracle into CSV File

I have written this code in C# to connect to an Oracle Database, omitting the tnsnames.ora file, then place results from a query into a CSV File. Is there more efficient way to do this? (Such as ...
8
votes
3answers
237 views

Calculating effective shop inventories from CSV files

Program purpose The purpose for the program is to create a file containing the effective inventory for each stocked item stored in a collection of shops. Input and algorithm details For clarity, ...
5
votes
1answer
177 views

Convert IEnumerable<Reads> to CSV

I have an IEnumerable of a view model that I need to convert to CSV and send back as a file response. I have written it out quite manually and I'm just seeking code ...
4
votes
2answers
587 views

Exporting a built data set to a CSV

I wrote a program to scan files, extract text, build a data set and export it to a CSV. My initial program was ugly, but it worked. I have since refactored it, but the latest version seems to run a ...
6
votes
2answers
164 views

Program to calculate some stats from Facebook messages

I've written a program that parses JSON downloaded from Facebook to calculate some stats from a conversation between two parties. It attemps to count these things: Messages grouped by day of week ...
3
votes
1answer
98 views

CSV Reader from scratch

I am new to Java and not quite familiar with its design patterns. I have tried to implement a CSV-File reader from scratch. The CSVFile constructor accepts the path to the file and a class which ...
2
votes
2answers
58 views

Parse and sort domain names and their prices

I set out to write a small Python script to tell me cheapest domains. I manually scraped for data from iwantmyname.com into a plain text file as follows: ...
0
votes
1answer
1k views

Multiply two columns of Census data and groupby

I have census data that looks like this ...
3
votes
1answer
55 views

Code runs Speed Test CLI using the python wrapper and then stores values within CSV

I wanted to make use of the speedtest-cli while also incorporating pandas into the project. My initial goal, which I am posting here, was to take the values of the speed test and incorporate them into ...
2
votes
0answers
30 views

Determine file provider based on column headers, replace columns and process data

Each day I receive files from different customers relaying the same information, but the customers use proprietary formats. For example, ...
5
votes
2answers
576 views

String Similarity using fuzzywuzzy on big data

I have a file in which I was to check the string similarity within the names in a particular column. I use fuzzywuzzy token sort ratio algorithm as it is required for my use case. here is the code, is ...
3
votes
2answers
76 views

Replacing values in a list

This is my first Powershell script ever so bear with me. I'm trying to replace values in a sharepoint list from a CSV file: ...
5
votes
1answer
60 views

Search for subsets within a 250k line text dataset

The code below works pretty well - it takes about 4 seconds (this seems too slow for me) to find a group that's located in the tail area of the text file I'm searching. The text file has ≃ 250,000 ...
0
votes
2answers
129 views

Removing entries with duplicate columns from large .vcf files

First of all, please note that the input (.vcf) file is very large (>60 GB). So I am reading the .vcf files line by line, but since I have to check for duplicate of two columns present in the input ...
8
votes
3answers
345 views

Parsing Excel file into CSV file

EDIT: Thanks everyone for your answers, this is moving quickly now and it's always essential to get better with design and best practice! I am loading in an excel file to a dataset using ...
0
votes
1answer
760 views

Generate a CSV file from a multidimensional array in php

I can tell by looking at this code that I've gone about this in the wrong way. Provided the string is correctly formatted (each index ofCSVData[][], would the ...
4
votes
4answers
113 views

Export dataset to CSV in C

I've just learnt the basics of C and I've built a working program that reads the dataset provided by my lecturer and exports it out to a CSV. However, he said that this could've been simplified (a ...
4
votes
1answer
77 views

Munging select rows in CSV files using Pandas

I'm learning Pandas. I have a project where I need to munge some CSV files and resave them as CSV. I can use dictionaries and CSV module, but decided to use DataFrames to get more exposure and ...
0
votes
0answers
43 views

Writing a table of national park data

I'm working to produce a Lookup table using the Arcpy module that contains descriptive statistics for US National Parks. The original data, found in tbl, has 12 ...
5
votes
2answers
67 views

Transpose tab-delimited table in C

I have looked online for transposing programs, but have found that they have extreme RAM use for large tables. The RAM use is so extreme I can't run most jobs on my laptop. I also have a Perl ...
3
votes
5answers
1k views

Code to initialize a .NET object from a CSV file

I am initializing the following object from a CSV file: ...
6
votes
1answer
137 views

Locating contact information from a CSV spreadsheet

I'm trying to find a more effective way to access variables within for loops. I felt that my attempt which works is not an effective way to access variables that ...
3
votes
0answers
48 views

Drupal 8 Content Import via CSV Implementation

I am implementing a way to import content into Drupal 8 via a CSV. This CSV contains various fields such as node ID, title, body, etc. It has been working great but I am very concerned about how long ...
2
votes
1answer
44 views

Consolidate multiple very large medical claim response text files for database load

Each month we receive 30 text files containing various medical claim responses. We sort these into three subfolders and then consolidate and load them into a database table for analysis. This may not ...
4
votes
1answer
41 views

Parsing positive integers from CSV file Part-2

Part-1 here This is a small CSV parser for positive integers mostly intended for personal projects. I (hopefully) improved the code from Part-1 by condensing some parts and improving others while ...
4
votes
0answers
401 views

CSV Data Table Extension

This is my library I wrote from scratch for the .net core that imports and exports CSVs from DataTable objects. I've seen a lot of huge complicated libraries that handle this and a million other ...