The following is a python program processing files input and generating a .csv
output. There are two input files of .txt
format, whose main body are of the format as follows:
video_name method_name
criteria_1_result(Named 'AUC' criteria)
criteria_2_result(Named 'DP' criteria)
And two final average result lines and a may-not-exist EOF indicator(>>
) are at the end.
And what I want to do is to generate a .csv
file to compare results of the two criteria for different method and different videos. And I used a dict whose key is the video sequence name and whose value is another dict containing what I what to save to the .csv
file.
Any suggestions to improve this small program?
Sample input:
1.txt:
video1 method1
10.0
11.0
video2 method1
20.0
9.0
15.0
10.0
>>
2.txt
video3 method2
10.0
11.0
video2 method2
20.0
9.0
video1 method2
15.0
10.0
15.0
10.0
Sample output:
"Seq","AUC_1","AUC_2","AUC_Diff","DP_1","DP_2","DP_Diff"
"video1","10.0","15.0","5.0","11.0","10.0","1.0"
"video2","20.0","20.0","0.0","9.0","9.0","0.0"
"video3","-","10.0","-","-","11.0","-"
My code:
# -*- coding: utf-8 -*-
# Created At 2017-09-18 10:34
# Version: 1.0
# License: GPL 2.0
# Description:
# This program read two input files, saving data and difference
# to a csv file for comparision
import csv
import math
def read_file(f_name):
"""
Read and get file content,
remove redundant '>>' at the end if exists
:param f_name: Name of the file to read
:return: List of lines of the file
"""
with open(f_name, 'r') as f:
content = f.readlines()
if content[-1].strip() == ">>":
content = content[:-1]
return content
def fill_record(f_content, record, csv_headers, idf):
"""
Using content read from input file, fill 'record' dict.
Dict keys are specified from 'csv_headers'.
'idf' indicates the suffix of columns like '1' in 'AUC_1'
:param f_content: list of input file lines
:param record: dict for saving data
:param csv_headers: list specifying dict keys
:param idf: '1' or '2', indicating the suffix like '1' in 'AUC_1'
:return: return dict filled with data from both input and calculated
"""
it_f = 0
while it_f < len(f_content)-2:
seq_name = f_content[it_f].strip().split(' ')[0].lower()
if seq_name not in record:
record[seq_name] = {}
for column in csv_headers:
record[seq_name][column] = '-'
record[seq_name]['AUC_'+idf] = f_content[it_f+1].strip()
record[seq_name]['DP_'+idf] = f_content[it_f+2].strip()
if record[seq_name]['AUC_1'] != '-' and record[seq_name]['AUC_2'] != '-':
record[seq_name]['AUC_Diff'] = str(math.fabs(float(record[seq_name]['AUC_1']) -
float(record[seq_name]['AUC_2'])))
record[seq_name]['DP_Diff'] = str(math.fabs(float(record[seq_name]['DP_1']) -
float(record[seq_name]['DP_2'])))
it_f += 3
return record
def write_to_csv_file(record, csv_headers, compare_result_name):
"""
Writing data to .csv files.
:param record: Dict filled with data to write
:param csv_headers: Dict keys / CSV file columns
:param compare_result_name: file name for output .csv file
:return: None
"""
with open(compare_result_name, 'w') as csvf:
writer = csv.writer(csvf, quoting=csv.QUOTE_ALL)
writer.writerow(csv_headers)
for seq in record:
row = ['-' for i in range(7)]
row[0] = seq
for i in range(len(csv_headers[1:])):
row[1+i] = record[seq][csv_headers[1+i]]
writer.writerow(row)
def main():
"""
Main function
:return: None
"""
# Change the input file names and output file name in need
f1_name = '1.txt'
f2_name = '2.txt'
compare_result_name = 'compare.csv'
csv_headers = ['Seq', 'AUC_1', 'AUC_2', 'AUC_Diff', 'DP_1', 'DP_2', 'DP_Diff']
record = {}
f1_content = read_file(f1_name)
f2_content = read_file(f2_name)
record = fill_record(f1_content, record, csv_headers, '1')
record = fill_record(f2_content, record, csv_headers, '2')
write_to_csv_file(record, csv_headers, compare_result_name)
if __name__ == "__main__":
main()