Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
-2
votes
1answer
41 views

what is the best practice concerning opening CSV files in python [closed]

I am looking for the best way to save regular data from a series of sensors to a CSV file. the program will be reading every sensor once a minute 24/7 and needs to report the results to a CSV file ...
5
votes
3answers
105 views

Creating a CSV stream from an object

I have this code that gets a CSV string from a list of objects where the first row in the string is the object's properties and the rows underneath are the properties values. I then create a ...
6
votes
3answers
161 views

CSV to HTML Converter

I have written a little program that converts a CSV-File to an HTML-Table. It works for my purposes. But are there parts in my code that can be written more clean? Can you improve maybe the ...
2
votes
2answers
45 views

Go Quiz exercise, part of Gophercises

This is the first exercise from https://gophercises.com/. Basically the idea is to parse a CSV file in a form of 'question:answer' and count the number of correct answers. Also you have a time out ...
3
votes
0answers
47 views

Reorder the Columns in a CSV File in Descending Order

I wrote a script to reorder the columns in a CSV file in descending order and then write to another CSV file. My script needs to be able to handle several tens of millions of records, and I would like ...
1
vote
0answers
29 views

How can this JavaScript CSV to JSON hook code be optimized to prevent timeouts?

This code is utilized within a platform that has strict limitations on script execution time, occasionally being exceeded if there is a very large number of lines within the CSV input that is passed ...
5
votes
1answer
126 views

Concatenate several CSV files in a single dataframe

I have currently 600 CSV files (and this number will grow) of 50K lines each i would like to put in one single dataframe. I did this, it works well and it takes 3 minutes : ...
3
votes
1answer
67 views

Sending a CSV file for a client list from a database

What can I improve about this code in order to be production-ready? I'm not worried about security but about errors that could occur. What exceptions should I catch? I feel overwhelmed about ...
3
votes
1answer
36 views

Count the number of riders in each category

I have a code to get riders from certain teams and calculate the number of riders in those teams in that category: ...
2
votes
1answer
79 views

Library for parsing CSV data

Please review my C++ CSV parsing class. I have some specific questions: Should get_next_record be a static function? CsvParser ...
3
votes
1answer
78 views

C# Reflection-based CSV Parser

I've been trying to make a C# version of my Java CSV Parser using C# specific idioms. Here is the full code: ...
3
votes
2answers
90 views

Java Reflection-based CSV Parser

I'm an experienced Java developer, however I don't have experience with reflection or Annotation classes. For fun, I tried to develop a CSV Reader class which can parse each row into a particular type....
3
votes
1answer
71 views

Calculating the total daily amount of confirmed cases of Coronavirus

I'm writing a small program to plot new COVID-19 infections. As of right now, I have it so the program reads the given data file, pulls out the daily cases and dates for each country, and adds ...
4
votes
1answer
70 views

C++ - a header only library for CSV

I started to learn C++ after programming for some years in Java, and this is my first (header-only) library. When switching from another language, it is always difficult not to project the old habits ...
1
vote
0answers
39 views

Converting multiple HTML files to csv lowers down the speed

New to python. I am converting multiple html files to csv. Developed a full code. It can easily convert 50+ files at a time but after 200+ files, the code takes too much time and with 1000s to ...
3
votes
1answer
153 views

Converting JSON to CSV using powershell

I have a very large JSON Lines File with 4.000.000 Rows, and I need to convert several events from every row. The resulted CSV File contains 15.000.000 rows. How can I optimize this script? I'm using ...
7
votes
2answers
270 views

Rust CSV Sorter Code Lookover

I am new to Rust, began learning a few days ago. I have written a simple csv_sorter based off of one I had written for a class previously. Everything runs fine and I have gotten my expected results. I ...
2
votes
3answers
165 views

Efficiently convert 60 GB JSON file to a csv file

Description Simply take a JSON file as input and convert the data in it into a CSV file. I won't describe the functionality in too much detail since I have reasonable docstrings for that. As you can ...
1
vote
1answer
73 views

I made a Python program to calculate price based on Inflation Rate

I made a program that calculates how much the value of money has decreased due to inflation over a given set of years. This takes a principal amount, a start and end date, and with the help of the ...
2
votes
2answers
61 views

Receive data from SQL and CSV and send it to MySQL database using Python

Details: I have a code, which does the following: Get data from CSV and insert it into MySQL table Get data from SQL Server and insert it into MySQL table Run this code every 240 seconds - to ...
2
votes
1answer
68 views

Parse org-mode clocktable CSVs into time sheets

For my student job, I have been logging work times with the org-mode in emacs for quite some time. Now since I can only work from remote, I figured it would be nice ...
3
votes
1answer
64 views

Summing CSV files row by row using Node.js events

I need to read multiple CSV files that have the same number of rows, and compute some results for each row. For simplification purposes, and in the scope of this code review, this computation would be ...
1
vote
3answers
60 views

Scraping the Bundesliga table and saving it to CSV

Wrote my first mini-script to get data from the german football league table and save the data to a csv file. Do you like the approach or could I be more efficient? Thanks in advance! ...
9
votes
3answers
202 views

Performance - Read large amount of XMLs and load into single csv

I am dealing with a large amount of XML files which I obtained from here https://clinicaltrials.gov/ct2/resources/download#DownloadAllData. The download yields around 300.000 XML files of similar ...
0
votes
1answer
39 views

PHP - Importing columns from uploaded CSV file to SQL database (Laravel 5.7) [closed]

The goal is to upload the columns from the csv into the SQL Pro database. I am able to select the column names that have been uploaded from the csv file, the columns from the csv file are then ...
0
votes
2answers
66 views

Read .csv files and convert the rows into dict for looping

This script tries to parse .csv files and returns an iterable object to fetch one row at a time to do some process. It works fine but I think it needs improvements ...
2
votes
2answers
103 views

Parsing CSV header for validation [closed]

I have a simple application that reads a couple of csv files and outputs the input into catalogues for the website. However, users have a habit of opening the csv files and inserting extra columns ...
2
votes
0answers
146 views

application to get git commit and write data into csv file

I am writing a small application to get commits between two tags from different env (apple, banana) and write the result into a csv file. For security, I have to replace some sensitive information ...
3
votes
1answer
59 views

Get data from pipedrive API and insert it into Snowflake

I am a new python programmer. I have created the code to put API data from pipedrive to Snowflake database. Here are the steps in my code. Delete the csv file if it exists. Make an API call put all ...
9
votes
2answers
1k views

Are there ways to speed up this string fuzzy matching in Golang?

I have a piece of python code doing fuzzy matching which works very well and is pretty fast. For reference, it uses the following files: https://raw.githubusercontent.com/datasets/s-and-p-500-...
1
vote
1answer
76 views

File size when parsing XML

The code below takes a directory of XML files and parses them into a CSV file. Right now parsing around 60 XML files is fast and the output is a CSV file that is around 250MB. That is a really big ...
4
votes
1answer
368 views

Reading multiple csv files in a single dataframe

I have a lot of compressed csv files in a directory. I want to read all those files in a single dataframe. This is what I have done till now: ...
3
votes
0answers
50 views

Powershell optimization (scripts to run simultaneously)

I need some help here, I posted on stackoverflow and was directed here, just to put things in context, I am new to PowerShell, I have a task, that in simple terms, would take a csv with more than ...
3
votes
2answers
85 views

Validating CSV headers against an expected list

My application (legacy code) reads a CSV input file with an expected format with regards to Column names as follows: 1st column - Should say "Marker" 2nd column - Should say "Category" 3rd column ... ...
1
vote
0answers
46 views

Optimized AWS Lamda purpose [closed]

I'm looking for optimized code with faster execution time. This writes CSV to parquet with the max size of 64 MB chunks. we are trying to concatenate the csv file contents being written as an parquet ...
0
votes
1answer
29 views

Correctness and Performance of graph plotting on R

Current State: I have two CSV files, their sample attached below: FileOutputWithoutBuffer-Metrics.csv ...
7
votes
2answers
500 views

Fastest way to get a value from csv file using java 7

My code picks a value from a too large csv file (~300MB.) at a given index i and j (assuming the csv as 2D array because it ...
6
votes
1answer
123 views

Rust code that reads two columns of data from .txt file

I'm writing a code to read data from text file in Rust. The target files have two columns and non-fixed length rows, for example, example.txt 1 1 2 4 3 9 4 16 ...
4
votes
3answers
154 views

Finding indices of CSV headers in Java

I'm working on a project where I needed to extract the indices for the headers of a CSV file to initialise objects. I thought that I should work on my documentation process and learn how to document ...
8
votes
2answers
708 views

Analyzing records of London Underground outages

You have a file that describes historic data of the London Underground showing dates when stations have been unavailable. The file contains these fields, separated by a pipe character: Date: ...
29
votes
5answers
3k views

Python web-scraper to download table of transistor counts from Wikipedia

I have been looking for answers for how to easily scrape data from Wikipedia into a CSV file with Beautiful Soup. This is the code so far. Is there an easier way to do it? ...
2
votes
1answer
240 views

Multithreading to process requests and save results in python

I was presented with a task to come up with a script that generates a CSV with POSTAL codes via bruteforce (I'm in Brazil, so that means CEP to us). Points to note: I'm using an external library, ...
2
votes
1answer
66 views

Remapping property IDs when inserting CSV data to a database table

My code is functioning the way I'm expecting, with the if. However, it is not pythonic to me, and that bugs me. The multiple if in the code below are supposed to ...
2
votes
1answer
506 views

Exporting data from PostgreSQL as CSV to S3 bucket

I have a general question regarding DRY. As you can see there are several parts such as io.StringIO(), csv.writer, ...
3
votes
0answers
312 views

JSON to CSV in python using json.loads and json_normalize

I am trying to convert a JSON file to CSV format using Python. I am using the JSON.loads() method and then using json_normalize()...
2
votes
0answers
40 views

C++ Sql-like base operating on csv files

I implemented very constrained trivial implementation of sql querries operating on user files (in csv format). Where possible i tried to use modern C++ features. The fancy goal is to serve MySql ...
2
votes
1answer
61 views

Loading in two CSV files and storing them in two separate arrays

As the title says, I'm loading (or at least trying to) two CSV files and store them in two separate arrays. What I have so far worked, but it isn't the most elegant, no efficient solution. The end ...
4
votes
3answers
118 views

Functional CSV Parser

I've been starting to migrate some C++ code from imperative to more of a functional style. I'm new to functional programming so I'm trying to get a sense of how well this code fits into the functional ...
0
votes
1answer
920 views

Excel table filling with EPPlus

I am making a tool to read 3 input files in CSV and then filling an XLSX file with the information from the CSV. I want to know if I am repeating myself and if I can make it better and faster. ...
2
votes
1answer
41 views

Batch retrieve formatted address along with geometry (lat/long) and output to csv

I have a csv file with 3 fields, two of which are of my interest, Merchant_Name and City. My goal was to output multiple csv ...

1
2 3 4 5
12