Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

2
votes
0answers
18 views

Generate CSV from multiple files on Regex Python Output

I have written a python script using regex to extrapolate a poorly formatted text file data table into an actual data table. I am currently printing the output using . C:\my file location\file.csv in ...
-4
votes
0answers
20 views

How to export Mysql request on PHP to CSV file on UTF-8 without BOM character [closed]

I would like to extract my request SQL from php to a CSV file. When I write my csv file, I've an file encoding on UTF-8 with BOM. But When I open this file on excel, I modify the data and save. Excel ...
1
vote
1answer
41 views

Upload csv file to sqlserver with python

I have to weekly upload a csv file to sqlserver, and I do the job using python 3. The problem is that it takes too long for the file to be uploaded (around 30 minutes), and the table has 49000 rows ...
2
votes
1answer
28 views

json2CSV fails because of lack of RAM memory

I run this code on my Heroku Node.js server in order to get CSV for few hundred rows, but it fails with R14 - Memory Quota Exceeded. It worked fine when there were ...
1
vote
0answers
27 views

Python Spreadsheet Header Indexer

At work, I find myself frequently crunching CSVs with headers prone to change names and location. I have a solution, but with no opportunity for code review I was hoping SE could offer a look ...
3
votes
1answer
36 views

Excel to CSV converter

To check out the cssv module i wrote a small programm which takes all xlsx files in a folder and converts them into csv files. Let me know what you would improve on this program. xlsx_tp_csv ...
2
votes
0answers
29 views

Speed up iteration for large dataset in python

I'm trying to parse data into a csv prior to a bulk transaction into Neo4j. I'm using vast amounts of data in the relationships and wondered if anyone could help in speeding up the transactions below....
6
votes
3answers
172 views

Code to read and write CSV files

Below is my code that 1) writes a CSV file with three columns of integer data (plus column names on the first line) and 2) reads the CSV file. I'm new to C++ and would appreciate some code review. ...
7
votes
2answers
355 views

Grouping comma-separated lines together

I have comma-delimited files like these, where the first field is sorted in increasing order: Case 1 ( 1st file ) : ...
18
votes
4answers
3k views

Web scraping the titles and descriptions of trending YouTube videos

This scrapes the titles and descriptions of trending YouTube videos and writes them to a CSV file. What improvements can I make? ...
4
votes
2answers
76 views

Converting fixed-width text file to CSV in C

I'm trying to get reacquainted with C after years of higher-level languages. I wrote a small fixed-width to CSV converter. It works (for ASCII at least) and it's quite fast, but I wonder if I missed ...
2
votes
0answers
43 views

Detecting line separators and line breaks in Python CSV [closed]

Here is how I'm detecting a field separator and line break in a CSV file for which I do not already know the format. Does this look sufficient, or are there other things I should be adding here? <...
2
votes
3answers
50 views

Select data from csv and make associative array

I've got a .csv file with ZIP codes containing the city and its sub-districts 67401,City 1,district_a 67401,City 2,district_b 67401,City 3,district_c 67401,City 3,district_d 67401,City 3,district_e ...
2
votes
1answer
49 views

Reflection optimization for export CSV on large scale

So, I'm building export/import CSV helper. I have some performance issues in the code below. it takes me to parse CSV of 25,000 rows at 7 seconds. If someone can help, it will be awesome! ...
2
votes
1answer
235 views

CSV file download in Spring

I recently worked on a module, in which I implemented the functionality of downloading the CSV files. ...
3
votes
1answer
68 views

Simple fluent library to validate text content

Before I can perform a more or less complex mappings, treatments in business objects and saving results in databases, I often need to validate contents from flat text files or excel files All the ...
1
vote
1answer
77 views

Generate xyz coordinates of a tube made of multiple rings

In this code, I generate the coordinates of a tube made of multiple rings. Finally, I write those x,y,z coordinates in a .csv file. I want to simplify this code (maybe using a for loop?). ...
1
vote
0answers
34 views

Handling slow processing the Spring batch way

We are using spring-batch to extract data from a SAP system, during processing we pass the extracted full texts to a tagging service, in the end the enriched documents are stored in Solr to be served ...
2
votes
2answers
164 views

Comparing columns from two CSV files - follow-up

This is the second part of a question, you can find the first part here: Comparing columns from two CSV files I've made some changes to the script and here is what it looks like now: ...
1
vote
1answer
140 views

Comparing columns from two CSV files

This question has a second part here: Comparing columns from two CSV files - follow-up I'm currently trying to compare two CSV files column by column but only when their indexes match. Here is what ...
0
votes
2answers
187 views

Laravel: Export customer record as CSV

Using Laravel, I need to export some value from the DB in a CSV and then upload the CSV via sFTP or return it in a Response. I'm trying to be use S.O.L.I.D. principles but with this scenario, I'm not ...
0
votes
1answer
62 views

Taking text from a file and formatting it

My code takes numbers from a large text file, then splits it to organise the spacing and to place it into a 2-dimensional array. The code is used to get data for a job scheduler that I'm building. <...
-1
votes
1answer
48 views

Remove lines from csv if duplicate value in column [closed]

i am trying to write csv parser so if i have the same name in the name column i will delete the second name's line. For example: ...
2
votes
2answers
134 views

Split CSV file into a text file per row, with whitespace normalization

I have a CSV file that I would like to split up by row, do some whitespace processing on the string data in each row of the document column, and then output the processed data into separate TXT files ...
1
vote
0answers
29 views

Leavers Script to disable users in AD, Remove Groups, delete profile and move home folders

i am pretty new to powershell and im not great at coding, but i have managed to cobble together code from around the net to help save time when removing old students accounts in AD. Code currently ...
2
votes
1answer
48 views

CSV-simplified: A small open source project

I'm working on a small open source project called CSV-simplified for a course I'm taking (Bloc), and I'm looking for some feedback and potential collaborators. I don't actually know an programmers, so ...
1
vote
2answers
52 views

Stitching together badly formatted log files

I needed to stitch together a large number of badly formatted log files. I [painfully] developed a script for it in R, one that successfully dealt with the various flaws in the dataset. However, this ...
0
votes
1answer
105 views

Converting employee survey data from CSV to JSON

This Java code project was submitted for a job opportunity but was marked down for showing "bad habits that could be difficult to unlearn" but I am mystified by what this means. Any takers please. How ...
2
votes
1answer
289 views

Hotel booking system help on how to book nights and memberships

I am doing a GCSE NEA project for practice. Customer Loyalty Scheme The managers of Crawdale Hotel Group have decided to update their customer loyalty scheme. Customers joining the scheme ...
0
votes
1answer
71 views

Sorting CSV table in Powershell

I'm wondering if some PS gurus can advise on whether my method for sorting and recombining some data fields by using an array of PSCustomObjects is as efficient as it might be. I'm importing a CSV ...
4
votes
0answers
480 views

Python CSV to XML converter

I'm creating an application that reads in data from a CSV file and creates an XML file using LXML. The below code works as expected. However, before I develop it further I would like to refactor it ...
2
votes
1answer
59 views

Read a directory of CSV files in R

I'm writing a lot of R in my spare time and am trying to get into good habits by using TidyVerse as much as possible. A large part of this has required reading entire directories of dated log files ...
1
vote
1answer
46 views

CSV file cell slicer

This program is supposed to find and slice any cell within a CSV file which contains more than a defined number of characters. The files can be pretty large, so being a beginner I would like to know ...
2
votes
1answer
28 views

Sorting files into subdirectories based on keyword scores

This script sorts files from current directory into known subdirectories. it is built for speed because the keyword database used can be up to 20kb. each line of the database has ...
4
votes
2answers
132 views

Calculating travel statistics based on CSV data

I took a code test for acme transport company and was rejected. That's ok. I'm still learning how to code. I didn't get much feedback so that's a bit frustrating. Can anyone here provide tips on how ...
5
votes
1answer
67 views

Personal School Course Scheduler

A good friend of mine had the challenge of trying to build a schedule using all available times and classes through a spreadsheet... by hand. He asked me if I could build a program to generate valid ...
3
votes
1answer
60 views

Get owner of ~5 million files and folders using Powershell and Robocopy

I've written a Powershell script that uses Robocopy to get the full path of all files on a drive and then run it through a foreach loop to get the owner for each file and generate it in a CSV. The ...
3
votes
0answers
610 views

Fastest way to write large CSV file in python

I'm fairly new to python and pandas but trying to get better with it for parsing and processing large data files. I'm currently working on a project that requires me to parse a a few hundred CSV CAN ...
-3
votes
1answer
110 views

Format data for a CSV item

This function formats the text in each cell of a CSV. The goal of the function is to make sure that the sequence of characters in text will never break the cell's ...
4
votes
2answers
155 views

Iterate over a list of list names as file names

Interrogating a web API using query-url's, for each query I can get either zero hits, one hit, or multiple hits. Each of those categories needs to go into a separate CSV file for manual review or ...
6
votes
2answers
783 views

Split big file into separate files depending on column's distinct values

I have a big file, which contains column names in the first row. The data is about 20GB and consists of 10s of millions of rows. Every other row contains data which I would like to append to other ...
8
votes
2answers
51 views

Streaming scheduled TV shows according to a CSV playlist

I'm very new to Python, and I've made a small program/script that will allow me to stream specific files at a given time. My idea is that I have a .csv file for each day, with one row for each movie/...
1
vote
0answers
32 views

Grouped by First Two Digits of Account_ID of ~240 million rows of Pandas DF

I have 40 CSV files, each containing ~6 million rows, and 2 columns. The columns are account_id and account_balance. The ...
7
votes
3answers
131 views

Convert binary raster file to text CSV

I have a binary file input.hgt from the Shuttle Radar Topography Mission. It contains a 3601✕3601 matrix, stored as 2-Byte big-endian integer numbers. My code ...
1
vote
1answer
153 views

Handling periods ('.') in CSV column names with Pandas

I'm reading and processing a fairly large csv using Pandas and Python 3.7. Header names in the CSV have periods in them ('full stops', Britons say). That's a problem when you want to address data ...
4
votes
2answers
103 views

Parsing contents of a large zip file into a html parser into a .csv file

I have some zip files somewhere in the order of 2GB+ containing only html files. Each zip contains about 170,000 html files each. My code reads the file without extracting them, Passes the ...
11
votes
3answers
779 views

Aggregate prescriptions per drug in a CSV file

I have a question that is similar in spirit to this previously asked question. Nonetheless, I can't seem to figure out a suitable solution. Input: I have CSV data that looks like (FICTIONAL ...
1
vote
1answer
95 views

Scraping content from yellowpage and write them to a csv file

I've written a script in python to scrape names and phone numbers of a certain search from yellowpage traversing multiple pages. The scraper has the ability to collect all the links connected to all ...
1
vote
2answers
72 views

Powershell function to search CSV logs for certain regexes

I have a parser that goes through two different logs, both .csv files, and checks for certain lines based off the regex code that I have chosen. This parser works fine, it just takes about a minute ...
2
votes
1answer
70 views

Convenience function for writing files in CSV, JSON, etc

The McCabe code complexity of the following function is 19 as found by pytest --mccabe test.py (from my toy-project mpu). While ...