Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
2
votes
2answers
79 views

Stitching together badly formatted log files

I needed to stitch together a large number of badly formatted log files. I [painfully] developed a script for it in R, one that successfully dealt with the various flaws in the dataset. However, this ...
-4
votes
0answers
27 views

Creating Python dictionary from csv with lists as values [closed]

I have a large .csv file of coordinates similar to this: I am trying to loop through it to create a dictionary where each unique value for "Index" is a key, and the corresponding value is a ...
4
votes
1answer
197 views

Initializing an Android activity by reading a CSV file with image IDs, names, and descriptions

I am using Java to create an Android app. I have the main activity read a CSV file, create objects, and then save them into a singleton class so the rest of the activities in the app can access them ...
3
votes
1answer
65 views

Sending a CSV file for a client list from a database

What can I improve about this code in order to be production-ready? I'm not worried about security but about errors that could occur. What exceptions should I catch? I feel overwhelmed about ...
3
votes
1answer
36 views

Count the number of riders in each category

I have a code to get riders from certain teams and calculate the number of riders in those teams in that category: ...
3
votes
1answer
63 views

C# Reflection-based CSV Parser

I've been trying to make a C# version of my Java CSV Parser using C# specific idioms. Here is the full code: ...
2
votes
1answer
70 views

Library for parsing CSV data

Please review my C++ CSV parsing class. I have some specific questions: Should get_next_record be a static function? CsvParser ...
3
votes
2answers
85 views

Java Reflection-based CSV Parser

I'm an experienced Java developer, however I don't have experience with reflection or Annotation classes. For fun, I tried to develop a CSV Reader class which can parse each row into a particular type....
3
votes
1answer
69 views

Calculating the total daily amount of confirmed cases of Coronavirus

I'm writing a small program to plot new COVID-19 infections. As of right now, I have it so the program reads the given data file, pulls out the daily cases and dates for each country, and adds ...
3
votes
1answer
68 views

Converting JSON to CSV using powershell

I have a very large JSON Lines File with 4.000.000 Rows, and I need to convert several events from every row. The resulted CSV File contains 15.000.000 rows. How can I optimize this script? I'm using ...
4
votes
1answer
61 views

C++ - a header only library for CSV

I started to learn C++ after programming for some years in Java, and this is my first (header-only) library. When switching from another language, it is always difficult not to project the old habits ...
1
vote
0answers
39 views

Converting multiple HTML files to csv lowers down the speed

New to python. I am converting multiple html files to csv. Developed a full code. It can easily convert 50+ files at a time but after 200+ files, the code takes too much time and with 1000s to ...
7
votes
2answers
270 views

Rust CSV Sorter Code Lookover

I am new to Rust, began learning a few days ago. I have written a simple csv_sorter based off of one I had written for a class previously. Everything runs fine and I have gotten my expected results. I ...
2
votes
3answers
111 views

Efficiently convert 60 GB JSON file to a csv file

Description Simply take a JSON file as input and convert the data in it into a CSV file. I won't describe the functionality in too much detail since I have reasonable docstrings for that. As you can ...
2
votes
1answer
65 views

Parse org-mode clocktable CSVs into time sheets

For my student job, I have been logging work times with the org-mode in emacs for quite some time. Now since I can only work from remote, I figured it would be nice ...
1
vote
1answer
44 views

I made a Python program to calculate price based on Inflation Rate

I made a program that calculates how much the value of money has decreased due to inflation over a given set of years. This takes a principal amount, a start and end date, and with the help of the ...
2
votes
2answers
55 views

Receive data from SQL and CSV and send it to MySQL database using Python

Details: I have a code, which does the following: Get data from CSV and insert it into MySQL table Get data from SQL Server and insert it into MySQL table Run this code every 240 seconds - to ...
9
votes
3answers
189 views

Performance - Read large amount of XMLs and load into single csv

I am dealing with a large amount of XML files which I obtained from here https://clinicaltrials.gov/ct2/resources/download#DownloadAllData. The download yields around 300.000 XML files of similar ...
3
votes
1answer
48 views

Summing CSV files row by row using Node.js events

I need to read multiple CSV files that have the same number of rows, and compute some results for each row. For simplification purposes, and in the scope of this code review, this computation would be ...
0
votes
2answers
66 views

Read .csv files and convert the rows into dict for looping

This script tries to parse .csv files and returns an iterable object to fetch one row at a time to do some process. It works fine but I think it needs improvements ...
1
vote
3answers
57 views

Scraping the Bundesliga table and saving it to CSV

Wrote my first mini-script to get data from the german football league table and save the data to a csv file. Do you like the approach or could I be more efficient? Thanks in advance! ...
0
votes
1answer
38 views

PHP - Importing columns from uploaded CSV file to SQL database (Laravel 5.7) [closed]

The goal is to upload the columns from the csv into the SQL Pro database. I am able to select the column names that have been uploaded from the csv file, the columns from the csv file are then ...
2
votes
2answers
93 views

Parsing CSV header for validation [closed]

I have a simple application that reads a couple of csv files and outputs the input into catalogues for the website. However, users have a habit of opening the csv files and inserting extra columns ...
2
votes
0answers
146 views

application to get git commit and write data into csv file

I am writing a small application to get commits between two tags from different env (apple, banana) and write the result into a csv file. For security, I have to replace some sensitive information ...
1
vote
1answer
75 views

File size when parsing XML

The code below takes a directory of XML files and parses them into a CSV file. Right now parsing around 60 XML files is fast and the output is a CSV file that is around 250MB. That is a really big ...
3
votes
1answer
54 views

Get data from pipedrive API and insert it into Snowflake

I am a new python programmer. I have created the code to put API data from pipedrive to Snowflake database. Here are the steps in my code. Delete the csv file if it exists. Make an API call put all ...
2
votes
1answer
4k views

get city and state using zip code, scrape in Python

I'm looking for any and all feedback on quality, style and efficacy of the code. If there's a simple way to get the header put in, I'd love to hear that. The code is used to take zip codes from a csv ...
5
votes
3answers
6k views

Writing to a csv file in a customized way using scrapy

I've written a script in scrapy to grab different names and links from different pages of a website and write those parsed items ...
9
votes
2answers
957 views

Are there ways to speed up this string fuzzy matching in Golang?

I have a piece of python code doing fuzzy matching which works very well and is pretty fast. For reference, it uses the following files: https://raw.githubusercontent.com/datasets/s-and-p-500-...
4
votes
1answer
238 views

Reading multiple csv files in a single dataframe

I have a lot of compressed csv files in a directory. I want to read all those files in a single dataframe. This is what I have done till now: ...
11
votes
2answers
1k views

Showing virus scan results from an API and a CSV file

I am utilizing an API module for interacting with virustotal.com in order to get AntiVirus results based of SHA256 hashes. The code I have is working but I feel like it can be improved greatly. I ...
3
votes
0answers
49 views

Powershell optimization (scripts to run simultaneously)

I need some help here, I posted on stackoverflow and was directed here, just to put things in context, I am new to PowerShell, I have a task, that in simple terms, would take a csv with more than ...
3
votes
2answers
71 views

Validating CSV headers against an expected list

My application (legacy code) reads a CSV input file with an expected format with regards to Column names as follows: 1st column - Should say "Marker" 2nd column - Should say "Category" 3rd column ... ...
1
vote
0answers
46 views

Optimized AWS Lamda purpose [closed]

I'm looking for optimized code with faster execution time. This writes CSV to parquet with the max size of 64 MB chunks. we are trying to concatenate the csv file contents being written as an parquet ...
2
votes
1answer
41 views

Batch retrieve formatted address along with geometry (lat/long) and output to csv

I have a csv file with 3 fields, two of which are of my interest, Merchant_Name and City. My goal was to output multiple csv ...
0
votes
1answer
29 views

Correctness and Performance of graph plotting on R

Current State: I have two CSV files, their sample attached below: FileOutputWithoutBuffer-Metrics.csv ...
7
votes
2answers
438 views

Fastest way to get a value from csv file using java 7

My code picks a value from a too large csv file (~300MB.) at a given index i and j (assuming the csv as 2D array because it ...
6
votes
1answer
119 views

Rust code that reads two columns of data from .txt file

I'm writing a code to read data from text file in Rust. The target files have two columns and non-fixed length rows, for example, example.txt 1 1 2 4 3 9 4 16 ...
8
votes
1answer
3k views

Python CSV to XML converter

I'm creating an application that reads in data from a CSV file and creates an XML file using LXML. The below code works as expected. However, before I develop it further I would like to refactor it ...
7
votes
1answer
4k views

Fastest way to write large CSV file in python

I'm fairly new to python and pandas but trying to get better with it for parsing and processing large data files. I'm currently working on a project that requires me to parse a a few hundred CSV CAN ...
4
votes
3answers
150 views

Finding indices of CSV headers in Java

I'm working on a project where I needed to extract the indices for the headers of a CSV file to initialise objects. I thought that I should work on my documentation process and learn how to document ...
8
votes
2answers
701 views

Analyzing records of London Underground outages

You have a file that describes historic data of the London Underground showing dates when stations have been unavailable. The file contains these fields, separated by a pipe character: Date: ...
29
votes
5answers
3k views

Python web-scraper to download table of transistor counts from Wikipedia

I have been looking for answers for how to easily scrape data from Wikipedia into a CSV file with Beautiful Soup. This is the code so far. Is there an easier way to do it? ...
2
votes
1answer
171 views

Multithreading to process requests and save results in python

I was presented with a task to come up with a script that generates a CSV with POSTAL codes via bruteforce (I'm in Brazil, so that means CEP to us). Points to note: I'm using an external library, ...
3
votes
0answers
274 views

JSON to CSV in python using json.loads and json_normalize

I am trying to convert a JSON file to CSV format using Python. I am using the JSON.loads() method and then using json_normalize()...
3
votes
2answers
84 views

Multi-line text reader (x64 ASM)

Building upon my previous question, I have come up with a method that'll search through a buffer and return a pointer to the end of the line. The primary purpose of the function to extract rows from a ...
2
votes
1answer
421 views

Exporting data from PostgreSQL as CSV to S3 bucket

I have a general question regarding DRY. As you can see there are several parts such as io.StringIO(), csv.writer, ...
2
votes
1answer
66 views

Remapping property IDs when inserting CSV data to a database table

My code is functioning the way I'm expecting, with the if. However, it is not pythonic to me, and that bugs me. The multiple if in the code below are supposed to ...
8
votes
2answers
811 views

Getting current weather data using OpenWeatherMap API

I would really appreciate if someone could take a look at this code. I am not very advanced, so ANY kind of feedback and critique would be precious to me. What this program is supposed to do is to go ...
7
votes
1answer
20k views

Updating a .csv file

I have a CSV file, call it csv_file. It has the following content: Username, Password name1, pass1 name2, pass2 ... I also have a dictionary, call it ...

1
2 3 4 5
12