Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
7
votes
2answers
5k views

String Similarity using fuzzywuzzy on big data

I have a file in which I was to check the string similarity within the names in a particular column. I use fuzzywuzzy token sort ratio algorithm as it is required for my use case. here is the code, is ...
3
votes
2answers
94 views

Replacing values in a list

This is my first Powershell script ever so bear with me. I'm trying to replace values in a sharepoint list from a CSV file: ...
5
votes
1answer
64 views

Search for subsets within a 250k line text dataset

The code below works pretty well - it takes about 4 seconds (this seems too slow for me) to find a group that's located in the tail area of the text file I'm searching. The text file has ≃ 250,000 ...
0
votes
2answers
255 views

Removing entries with duplicate columns from large .vcf files

First of all, please note that the input (.vcf) file is very large (>60 GB). So I am reading the .vcf files line by line, but since I have to check for duplicate of two columns present in the input ...
8
votes
3answers
3k views

Parsing Excel file into CSV file

EDIT: Thanks everyone for your answers, this is moving quickly now and it's always essential to get better with design and best practice! I am loading in an excel file to a dataset using ...
1
vote
1answer
2k views

Generate a CSV file from a multidimensional array in php

I can tell by looking at this code that I've gone about this in the wrong way. Provided the string is correctly formatted (each index ofCSVData[][], would the ...
4
votes
4answers
132 views

Export dataset to CSV in C

I've just learnt the basics of C and I've built a working program that reads the dataset provided by my lecturer and exports it out to a CSV. However, he said that this could've been simplified (a ...
4
votes
1answer
131 views

Munging select rows in CSV files using Pandas

I'm learning Pandas. I have a project where I need to munge some CSV files and resave them as CSV. I can use dictionaries and CSV module, but decided to use DataFrames to get more exposure and ...
1
vote
0answers
73 views

Writing a table of national park data

I'm working to produce a Lookup table using the Arcpy module that contains descriptive statistics for US National Parks. The original data, found in tbl, has 12 ...
5
votes
2answers
85 views

Transpose tab-delimited table in C

I have looked online for transposing programs, but have found that they have extreme RAM use for large tables. The RAM use is so extreme I can't run most jobs on my laptop. I also have a Perl ...
6
votes
6answers
7k views

Code to initialize a .NET object from a CSV file

I am initializing the following object from a CSV file: ...
6
votes
1answer
152 views

Locating contact information from a CSV spreadsheet

I'm trying to find a more effective way to access variables within for loops. I felt that my attempt which works is not an effective way to access variables that ...
3
votes
0answers
60 views

Drupal 8 Content Import via CSV Implementation

I am implementing a way to import content into Drupal 8 via a CSV. This CSV contains various fields such as node ID, title, body, etc. It has been working great but I am very concerned about how long ...
2
votes
1answer
49 views

Consolidate multiple very large medical claim response text files for database load

Each month we receive 30 text files containing various medical claim responses. We sort these into three subfolders and then consolidate and load them into a database table for analysis. This may not ...
4
votes
1answer
43 views

Parsing positive integers from CSV file Part-2

Part-1 here This is a small CSV parser for positive integers mostly intended for personal projects. I (hopefully) improved the code from Part-1 by condensing some parts and improving others while ...
4
votes
0answers
905 views

CSV Data Table Extension

This is my library I wrote from scratch for the .net core that imports and exports CSVs from DataTable objects. I've seen a lot of huge complicated libraries that handle this and a million other ...
3
votes
1answer
143 views

Parsing positive integers from CSV file

I Needed to parse some positive integers with the following constraints: If possible use C++ only (no C API) Needs to be able to process parsed data after each newline Should be reasonably fast ...
1
vote
1answer
113 views

Read integers from text file and write to CSV file

Background: I've recently started learning bash due to my new job in a VFX company. We backup all of our media to LTO tapes (one master and one clone). I was tasked with writing a script that split ...
2
votes
2answers
410 views

ASCII encryption program in Python

I have recently written a program that reads information from a CSV file and puts it into lists. An encryption sub procedure converts the characters to ASCII, adds 2 then converts them back to ...
3
votes
0answers
93 views

Filtering OTC market data

It's been one month I do programming in python. Hence, my experience is a bit limited. Here is what I've done so far. The program is only filtering OTC market data according to certain features. Could ...
0
votes
1answer
5k views

Convert Excel files to CSV

I wrote the following script to run through several excel files and format them before saving as a CSV for upload to a Quickbase app I'm creating. Each excel file is roughly 100k lines and the code ...
7
votes
2answers
3k views

Parse one table out of CSV with multiple titled tables with the Python CSV module

I'm looking to parse CSV files containing multiple tables using Python 3's csv module. These complex CSVs are not unlike the toy example below. My goal is to make ...
2
votes
1answer
77 views

Crap (from the Past) Scraper

I listen to a podcast called Crap from the Past. It's a radio show that airs weekly in Minneapolis and specializes in playing music from my childhood. The show has been on the air for ~26 years. I ...
6
votes
3answers
215 views

File handling in C

I've created an app to manage students' data which also supports file handling. The Student structure looks like this: ...
1
vote
1answer
515 views

Write millions of lines to a file - Python, Dataframes and Redis [closed]

I have the following code snippet that reads a CSV into a dataframe, and writes out key-values pairs to a file in a Redis protocol-compliant fashion, i.e. ...
6
votes
1answer
526 views

Read CSV and use bidirectional BFS to find the shortest connections between actors

I have a really huge file¹ with 17 million records in it. Here is a sample of the file: Actor Movie 1,2 2,2 3,1 4,3 2,3 ...
2
votes
2answers
15k views

Using Pandas to merge .csv files

Hope you can help me out with this one because it is really slow. Is there a way to do this without loading the whole .csv into memory? The thing is... I have files containing timeseries data with 10 ...
5
votes
1answer
114 views

Analyzing the U.S. Births dataset in Python

I'm a Data Science newbie and currently looking to improve my code. I was trying to calculate the total number of the following: The total number of births on each month The total number of births on ...
4
votes
3answers
240 views

Beginner solution to basic quiz exercise, a la Gophercises

I just started golang development about two weeks ago and recently finished the recommended introduction book. I'm now working my way through Gophercises - a sort of collection of exercises to ...
2
votes
0answers
891 views

Scrape webpage with Beautifulsoup and export relevant data to csv

I am looking for feedback on my code, how I can simplify some operations, use more efficient methods or other best practices. This is my first web scraping project. The goal here is to scrape a ...
7
votes
3answers
4k views

Compute mean, variance and standard deviation of CSV number file

I've written my first C++ application this week and I wanted to see if there's anything that aren't up to standards or better ways to do what I'm currently doing. For the goal of the exercise, let's ...
5
votes
1answer
202 views

Get GPS coords with Addresses from CSV

I have a script that takes addresses from a CSV, and tries to find the GPS coordinates. I am still learning Python, so comments as to general formatting (i.e. variable names, function names, etc.) ...
1
vote
1answer
7k views

Python script to update an existing column values in a csv from another two tables

I need to update two columns: feedID and OperatID of table#1.csv with 'feed description', 'Operate description' from other CSV ...
20
votes
4answers
6k views

System that manages employee data for managers

This code is for an employee management system. The goal that I am trying to reach is to allow managers to access data of their employees more easily. The data that is entered is saved to a CSV file ...
9
votes
1answer
111 views

Text menu to browse CSV data about seal species

I am writing in C++, I have completed a small program which parses CSV files from IUCN and presents the species data using menus. I used a lot of loops. It works, however it is buggy when processing ...
2
votes
1answer
904 views

Export running data from Garmin API to CSV

I'm interested in getting better. Current script does the job but I'm really unsure regarding Quality of overall structure & comments Error handling Possibility the add testing (what and how ...
11
votes
4answers
1k views

Parsing RFC 4180 CSV with GOTOs

One of my data-import tools needs to support CSV files. I thought that parsing CSV is such a simple task that I didn't want to use any any external libraries for that. So here is one more RFC 4180 CSV ...
10
votes
2answers
662 views

Searching for data from file1 in file2

In order to answer a given question on Ask Ubuntu I came up with two scripts for doing the following task: Compare two files for an occurrence of column no. 6 value in file1 occurs in file2. On ...
1
vote
1answer
109 views

Comparing two n×m matrices

I have to do a pairwise compare of a couple of text files. They have the following format: ...
5
votes
1answer
82 views

Optimize and accelerate bash script to transform XML to SQLITE

I have this script in which I read an XML and I pass it to CSV and at the end of the script I transform it into SQLITE ...
1
vote
0answers
49 views

ruby schema matching code

I tried to find the correspondences between columns in two datasets, which is called schema matching. I implemented the algorithm like a c-style program so that I could implement it easily but I have ...
11
votes
3answers
8k views

Convert all CSV files in a given directory to JSON using Python

I am attempting to convert all files with the csv extension in a given directory to json with this python script. I am wondering if there is a better and more efficient way to do this? Here is my ...
4
votes
2answers
1k views

Compare two CSV files In Ruby

I wrote my first program in Ruby that compares two CSV files, but I'm sure there are more efficient ways to do it. I tried using the Ruby CSV library at first, but it was unproductive. Please let me ...
0
votes
1answer
105 views

Reading Maps of strings and doubles from CSV

I want to have 2 classes responsible for reading dictionaries from CSV. The 1st class takes 2 columns from CSV and makes Map<String, String>. The 2nd class ...
1
vote
1answer
54 views

Verify that a CSV file contains a backup job for every prod job

This is a list of jobs in autosys. What I have to do is make sure that every job in PROD has a accompanying job in BACKUP. The jobs are formatted as ...
2
votes
2answers
93 views

Pulling data from a CSV and populate AD

I've been piecing this together little by little for a little over a week. I'm curious if there's a cleaner way to accomplish what this is doing. Basically, it's pulling data from a CSV, creating ...
4
votes
1answer
437 views

Parsing a CSV file with Python

My goal is to parse this CSV data: ...
2
votes
1answer
216 views

Scraping different categories using different links through proxy

I've written a script in python to scrape name, review_star and review_count by going through thousands of links (to different products) stored in a csv file using reverse search. As those links are ...
3
votes
0answers
3k views

Reading a csv file and making a histogram in Python using NumPy and Matplotlib

I have written the following code which reads a csv file that contains a bunch of words and their sentiment value. Words like abandon may have a value of -1, while words like progress and freedom have ...
3
votes
1answer
88 views

Fetch and append player statistics to an Excel spreadsheet

I am a new programmer in python, and I need a bit of help to structure my code. ...