Here is how I'm detecting a field separator and line break in a CSV file for which I do not already know the format. Does this look sufficient, or are there other things I should be adding here?

SEPARATORS=['\x00', '\x01',  '^', ':', ',', '\t', ':', ';', '|', '~', ' ']
LINE_TERMINATORS_IN_ORDER = ['\x02\n', '\r\n', '\n', '\r']

def get_csv_info(f):
    with open(f, 'r') as csvfile:
        line = next(csvfile)
        dialect = csv.Sniffer().sniff(line, SEPARATORS)
        for _terminator in LINE_TERMINATORS_IN_ORDER:
            if line.endswith(_terminator):
                terminator = _terminator
        return (dialect.delimiter, terminator)

('\x01', '\x02\n')

Note: using the normal csv.Sniffer without params fails most of the time I use it.


closed as off-topic by Grajdeanu Alex., Sᴀᴍ Onᴇᴌᴀ, l0b0, Alex L, vnp Dec 18 '18 at 23:34

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Code not implemented or not working as intended: Code Review is a community where programmers peer-review your working code to address issues such as security, maintainability, performance, and scalability. We require that the code be working correctly, to the best of the author's knowledge, before proceeding with a review." – Grajdeanu Alex., Sᴀᴍ Onᴇᴌᴀ, l0b0, Alex L, vnp
If this question can be reworded to fit the rules in the help center, please edit the question.

  • \$\begingroup\$ SEPARATORS=['\x00', ..., ':', ..., ':',...] - why two :? \$\endgroup\$ – Patrick Artner Dec 18 '18 at 6:48
  • \$\begingroup\$ There is only one case where you need a single byte, which is the line ending with \r. If it doesn't end with it then you need two bytes, no need to go through a list. \$\endgroup\$ – ChatterOne Dec 18 '18 at 8:43
  • \$\begingroup\$ @ChatterOne could you please clarify what you mean with that? For example, what should the LINE_TERMINATORS be instead? \$\endgroup\$ – David542 Dec 18 '18 at 16:18

Browse other questions tagged or ask your own question.