Here is how I'm detecting a field separator and line break in a CSV file for which I do not already know the format. Does this look sufficient, or are there other things I should be adding here?
SEPARATORS=['\x00', '\x01', '^', ':', ',', '\t', ':', ';', '|', '~', ' '] LINE_TERMINATORS_IN_ORDER = ['\x02\n', '\r\n', '\n', '\r'] def get_csv_info(f): with open(f, 'r') as csvfile: line = next(csvfile) dialect = csv.Sniffer().sniff(line, SEPARATORS) for _terminator in LINE_TERMINATORS_IN_ORDER: if line.endswith(_terminator): terminator = _terminator break return (dialect.delimiter, terminator) get_csv_info('/Users/david/Desktop/validate_headers/artist') ('\x01', '\x02\n')
Note: using the normal
csv.Sniffer without params fails most of the time I use it.