# Grouping comma-separated lines together

I have comma-delimited files like these, where the first field is sorted in increasing order:

### Case 1 ( 1st file ) :

abcd,1
abcd,21
abcd,122
abce,12
abcf,13
abcf,21


### Case 2 ( and another file like this ) :

abcd,1
abcd,21
abcd,122


What I want to do is convert the first file to like this :

abcd 1,21,122
abce 12
abcf 13,21


And similarly, for the second file like this :

abcd 1,21,122


Now, I wrote a very ugly code with a lot of if's to check whether the next line's string before the comma is same as current line's string so, if it is then do ....

It's so badly written that, I wrote it myself around 6 months back and it took me around 3-4 minutes to understand why I did what I did in this code. Well in short it's ugly, in case you would like to see, here it is ( also there's a bug currently in here and since I needed a better way than this whole code so I didn't sort it out, for the curious folks out there the bug is that it doesn't print anything for the second case mentioned above and I know why ).

def clean_file(filePath, destination):
f = open(filePath, 'r')
f.close()
curr_string = current_number = next_string = next_number = ""
current_numbers = ""
lines = data.split('\n')[:-1]
for i in range(len(lines)-1):
print(lines[i])
curr_line = lines[i]
next_line = lines[i+1]
curr_string, current_number = curr_line.split(',')
next_string, next_number = next_line.split(',')
if curr_string == next_string:
current_numbers += current_number + ","
else:
current_numbers += current_number    # check to avoid ',' in the end
final_payload += curr_string + " " + current_numbers + "\n"
current_numbers = ""
# For last line
if curr_string != next_string:
else:
# Remove the newline, add a comma and then finally add a newline
with open(destination, 'a') as f:



Any better solutions?

## migrated from stackoverflow.comJan 13 at 17:33

This question came from our site for professional and enthusiast programmers.

• Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers. – Mast Jan 13 at 19:02

1. To solve the grouping problem, use itertools.groupby.
2. To read files with comma-separated fields, use the csv module.
3. In almost all cases, open() should be called using a with block, so that the files will be automatically closed for you, even if an exception occurs within the block:

with open(file_path) as in_f, open(destination, 'w') as out_f:
# code goes here

4. filePath violates Python's official style guide, which recommends underscores, like your curr_line.

While @200_success's answer is very good (always use libraries that solve your problem), I'm going to give an answer that illustrates how to think about more general problems in case there isn't a perfect library.

### Use with to automatically close files when you're done

You risk leaving a file open if an exception is raised and file.close() is never called.

with open(input_file) as in_file:


### Use the object to iterate, not indices

Most collections and objects can be iterated over directly, so you don't need indices

with open(input_file) as in_file:
for line in in_file:
line = line.strip() # get rid of '\n' at end of line


### Use data structures to organize your data

In the end, you want to associate a letter-string with a list of numbers. In python, a dict allows you to associate any piece of data with any other, so we'll use that to associate the letter-strings with a list of numbers.

with open(input_file) as in_file:
data = dict()
for line in in_file:
line = line.strip() # get rid of '\n' at end of line
letters, numbers = line.split(',')
data[letters].append(numbers)


Now, this doesn't quite work since, if a letters entry hasn't been seen yet, the call to data[letters] won't have anything to return and will raise a KeyError exception. So, we have to account for that

with open(input_file) as in_file:
data = dict()
for line in in_file:
line = line.strip() # get rid of '\n' at end of line
letters, number = line.split(',')
try: # there might be an error
data[letters].append(number) # append new number if letters has been seen before
except KeyError:
data[letters] = [number] # create new list with one number for a new letter-string


Now, all of the file is stored in a convenient form in the data object. To output, just loop through the data

with open(input_file) as in_file:
data = dict()
for line in in_file:
line = line.strip() # get rid of '\n' at end of line
letters, number = line.split(',')
try: # there might be an error
data[letters].append(number) # append new number if letters has been seen before
except KeyError:
data[letters] = [number] # create new list with one number for a new letter-string

with open(output_file, 'w') as out_file:
for letters, number_list in data.items(): # iterate over all entries
out_file.write(letters + ' ' + ','.join(number_list) + '\n')


The .join() method creates a string from a list such that the entries of the list are separated by the string that precedes it--',' in this case.

• Instead of trying to append and catching the error, you can use setdefault: data.setdefault(letters, []).append(number) – Todd Sewell Jan 13 at 23:04
• @ToddSewell Neat! That'll be useful in the future. – Mark H Jan 13 at 23:08
• Or use collections.defaultdict of course. – Graipher Jan 14 at 14:24