Questions tagged [csv]

A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.

Filter by
Sorted by
Tagged with
7
votes
2answers
5k views

Parse one table out of CSV with multiple titled tables with the Python CSV module

I'm looking to parse CSV files containing multiple tables using Python 3's csv module. These complex CSVs are not unlike the toy example below. My goal is to make ...
3
votes
1answer
89 views

Crap (from the Past) Scraper

I listen to a podcast called Crap from the Past. It's a radio show that airs weekly in Minneapolis and specializes in playing music from my childhood. The show has been on the air for ~26 years. I ...
6
votes
3answers
227 views

File handling in C

I've created an app to manage students' data which also supports file handling. The Student structure looks like this: ...
1
vote
1answer
623 views

Write millions of lines to a file - Python, Dataframes and Redis [closed]

I have the following code snippet that reads a CSV into a dataframe, and writes out key-values pairs to a file in a Redis protocol-compliant fashion, i.e. ...
6
votes
1answer
581 views

Read CSV and use bidirectional BFS to find the shortest connections between actors

I have a really huge file¹ with 17 million records in it. Here is a sample of the file: Actor Movie 1,2 2,2 3,1 4,3 2,3 ...
2
votes
2answers
16k views

Using Pandas to merge .csv files

Hope you can help me out with this one because it is really slow. Is there a way to do this without loading the whole .csv into memory? The thing is... I have files containing timeseries data with 10 ...
5
votes
1answer
140 views

Analyzing the U.S. Births dataset in Python

I'm a Data Science newbie and currently looking to improve my code. I was trying to calculate the total number of the following: The total number of births on each month The total number of births on ...
4
votes
3answers
267 views

Beginner solution to basic quiz exercise, a la Gophercises

I just started golang development about two weeks ago and recently finished the recommended introduction book. I'm now working my way through Gophercises - a sort of collection of exercises to ...
2
votes
0answers
972 views

Scrape webpage with Beautifulsoup and export relevant data to csv

I am looking for feedback on my code, how I can simplify some operations, use more efficient methods or other best practices. This is my first web scraping project. The goal here is to scrape a ...
7
votes
3answers
5k views

Compute mean, variance and standard deviation of CSV number file

I've written my first C++ application this week and I wanted to see if there's anything that aren't up to standards or better ways to do what I'm currently doing. For the goal of the exercise, let's ...
5
votes
1answer
307 views

Get GPS coords with Addresses from CSV

I have a script that takes addresses from a CSV, and tries to find the GPS coordinates. I am still learning Python, so comments as to general formatting (i.e. variable names, function names, etc.) ...
1
vote
1answer
10k views

Python script to update an existing column values in a csv from another two tables

I need to update two columns: feedID and OperatID of table#1.csv with 'feed description', 'Operate description' from other CSV ...
21
votes
4answers
6k views

System that manages employee data for managers

This code is for an employee management system. The goal that I am trying to reach is to allow managers to access data of their employees more easily. The data that is entered is saved to a CSV file ...
9
votes
1answer
111 views

Text menu to browse CSV data about seal species

I am writing in C++, I have completed a small program which parses CSV files from IUCN and presents the species data using menus. I used a lot of loops. It works, however it is buggy when processing ...
2
votes
1answer
1k views

Export running data from Garmin API to CSV

I'm interested in getting better. Current script does the job but I'm really unsure regarding Quality of overall structure & comments Error handling Possibility the add testing (what and how ...
11
votes
4answers
1k views

Parsing RFC 4180 CSV with GOTOs

One of my data-import tools needs to support CSV files. I thought that parsing CSV is such a simple task that I didn't want to use any any external libraries for that. So here is one more RFC 4180 CSV ...
9
votes
2answers
782 views

Searching for data from file1 in file2

In order to answer a given question on Ask Ubuntu I came up with two scripts for doing the following task: Compare two files for an occurrence of column no. 6 value in file1 occurs in file2. On ...
1
vote
1answer
122 views

Comparing two n×m matrices

I have to do a pairwise compare of a couple of text files. They have the following format: ...
5
votes
1answer
87 views

Optimize and accelerate bash script to transform XML to SQLITE

I have this script in which I read an XML and I pass it to CSV and at the end of the script I transform it into SQLITE ...
1
vote
0answers
50 views

ruby schema matching code

I tried to find the correspondences between columns in two datasets, which is called schema matching. I implemented the algorithm like a c-style program so that I could implement it easily but I have ...
11
votes
3answers
9k views

Convert all CSV files in a given directory to JSON using Python

I am attempting to convert all files with the csv extension in a given directory to json with this python script. I am wondering if there is a better and more efficient way to do this? Here is my ...
4
votes
2answers
1k views

Compare two CSV files In Ruby

I wrote my first program in Ruby that compares two CSV files, but I'm sure there are more efficient ways to do it. I tried using the Ruby CSV library at first, but it was unproductive. Please let me ...
0
votes
1answer
122 views

Reading Maps of strings and doubles from CSV

I want to have 2 classes responsible for reading dictionaries from CSV. The 1st class takes 2 columns from CSV and makes Map<String, String>. The 2nd class ...
1
vote
1answer
56 views

Verify that a CSV file contains a backup job for every prod job

This is a list of jobs in autosys. What I have to do is make sure that every job in PROD has a accompanying job in BACKUP. The jobs are formatted as ...
2
votes
2answers
100 views

Pulling data from a CSV and populate AD

I've been piecing this together little by little for a little over a week. I'm curious if there's a cleaner way to accomplish what this is doing. Basically, it's pulling data from a CSV, creating ...
4
votes
1answer
466 views

Parsing a CSV file with Python

My goal is to parse this CSV data: ...
2
votes
1answer
237 views

Scraping different categories using different links through proxy

I've written a script in python to scrape name, review_star and review_count by going through thousands of links (to different products) stored in a csv file using reverse search. As those links are ...
3
votes
0answers
3k views

Reading a csv file and making a histogram in Python using NumPy and Matplotlib

I have written the following code which reads a csv file that contains a bunch of words and their sentiment value. Words like abandon may have a value of -1, while words like progress and freedom have ...
3
votes
1answer
88 views

Fetch and append player statistics to an Excel spreadsheet

I am a new programmer in python, and I need a bit of help to structure my code. ...
4
votes
1answer
2k views

Python CSV file slicer

I have made CSV file slicer. This is my first and biggest piece of code on Python. It takes one .csv file from current folder and then slices it to n parts and adds ...
4
votes
2answers
2k views

Scraper to deal with some complicated site with ads

I've written a script in Python in combination with Selenium to scrape data from a slightly complicated web page. When the web page is made to open, it comes up with an ad sitting on the data making ...
1
vote
0answers
12k views

Tkinter Application to Read & Update a CSV File

Background I started learning Python about 3 months ago. I feel like I learned a lot, but would like to receive some advices from the pros and some feedback about my code. So I wrote a small GUI app. ...
7
votes
1answer
8k views

CSV file reader in PHP that supports large files (>15k lines)

I have written the following function in PHP to read a CSV file. It works correctly for small files. However, if I try to read in files that are bigger than 15k lines, it takes between 1–2 ...
3
votes
2answers
79 views

Sort a file on the first two columns, with optional filtering

I am a newbie to Rust and low-level coding. I come from data science and I have always used scripting languages so I may have done weird or useless things (with lifetimes and references). This is ...
1
vote
1answer
44 views

script that imports only data that is not in the database yet

I want to write a script that imports data into a database (in django). The database has a Company class with a name (CharField) instance. Additionally, it has a Person class with name (CharField), ...
4
votes
1answer
131 views

Data comparing and difference calculating python program

The following is a python program processing files input and generating a .csv output. There are two input files of .txt format, ...
1
vote
1answer
107 views

Read in a file, check that it meets certain criteria

This was my original posting: Check that a data file contains the expected number of rows and columns of integers I updated the code. Still what could I improve on? Another question I have is how ...
1
vote
1answer
2k views

Check that a data file contains the expected number of rows and columns of integers

This is a class I wrote check to check the format of .dat files. My code makes sure that the content of the files satisfies the conditions. A file can be deemed invalid if: The file can't be found. ...
2
votes
1answer
62 views

Retrieving records from a CSV file with a geographical hierarchy

I have the following scenario and I would like some review or suggestions. I have a CSV file with lot of records. The records are basically as follows: There are names of counties in Kenya. These ...
1
vote
1answer
889 views

Extracting certain products from a webpage using Scrapy

I've written a script in Python Scrapy to harvest various product names and price from books.toscrape. The reason I submit this tiny code to Code Review is because, in Python 3 when it comes to work ...
2
votes
1answer
221 views

ETL project, handling XML, CSV, and text files

I have completed an ETL project to collect, parse and load files. I decided to make it clean OOP way using interfaces and abstract, but have some questions below. ...
7
votes
1answer
3k views

Count occurrences of date-time in large CSV

My input file is a CSV of varying size, up to around 10GB. The file has several fields, however I'm only interested in the third column, a date-time field. The date-times are in UTC timezone, and they ...
5
votes
1answer
96 views

Saving web search results for EANs to CSV

This is my first program in Python. Following code is taken from different posts (mostly this site) and combined together so that I can automate my routine task. It's working properly but I need ...
2
votes
1answer
36 views

linux rearange field and sort by column

I have (tab-separated) input files: id1 id2 .... id1 id3 .... id3 id4 .... id2 id1 .... id3 id4 .... ..... I need to ...
1
vote
2answers
8k views

Opening a CSV file in Python, with error handling

Trying to improve my function, as will be used by most of my code. I'm handling most common exception (IOError) and handling when data has no values. ...
2
votes
2answers
172 views

Finding football team with the smallest difference in goals

The file football.dat contains the results from the English Premier League for 2001/2. The columns labeled ‘F’ and ‘A’ contain the total number of goals scored for and against each team in that season ...
5
votes
3answers
1k views

WV lottery tool in Python

I am self taught in python. I love to play the lottery and play with numbers. I put this program together as a hobby to learn python and build something that people can use. I used and reused code ...
2
votes
1answer
153 views

Adding values to rows in CSV

This code below is for one of my scripts that I am trying to automate. I am dealing with the rows and all the data that is in those rows. What I am trying to do is make this look neater and maybe ...
5
votes
1answer
60 views

Aligning fstab columns

I know I can use the unix column function to replicate 90% of the functionality here, so this is mostly just to practice python. I find a misaligned fstab file really hard to read. This script parses ...
3
votes
2answers
25 views

Efficient way of reading in list of string csvs' into lists

I have a list of strings. Each string is a comma separated list of numbers. I want to quickly convert each of these strings into a list. So far I have done the following: My Solution So Far ...

1 2 3
4
5
12