Update the idx values if vals element match between consecutive rows

Question

I want to update the numbers in idx field if there is any matching letter between the vals from two consecutive row.

Input data = '''pos\tidx\tvals
    23\t4\tabc
    25\t7\tatg
    29\t8\tctb
    35\t1\txyz
    37\t2\tmno
    39\t3\tpqr
    41\t6\trtu
    45\t5\tlfg'''

Explantion: Since there is letteramatching between idx 4 and 7 the idx at pos 25 will be updated to 4, but again there istmatching between vals at pos 25 and 29 we update the idx at 29 also to 4 instead of just 7.

#Expected output to a file (tab separated):
pos    idx    vals
23    4    abc
25    4    atg
29    4    ctb
35    1    xyz
37    2    mno
39    3    pqr
41    3    rtu
45    5    lfg

I have written the given workable code (below) so far, and would also like to

• write the expected output to a file
• optimize the code for the work I am doing.
• the answer has to follow my method of reading two consecutive rows (as keys, values) pairs at a time in ordered way. The reason is this question is just a trial of other problem I am trying to solve. Other parts of the code can be optimized in any pythonic way.

Code:

import csv
import itertools
import collections
import io
from itertools import islice

data = '''pos\tidx\tvals
23\t4\tabc
25\t7\tatg
29\t8\tctb
35\t1\txyz
37\t2\tmno
39\t3\tpqr
41\t6\trtu
45\t5\tlfg'''


data_As_Dict = csv.DictReader(io.StringIO(data), delimiter='\t')
grouped = itertools.groupby(data_As_Dict, key=lambda x: x['idx'])

''' Function to read the data as key, val pairs in Ordered way.'''
def accumulate(data):
    acc = collections.OrderedDict()
    for d in data:
        for k, v in d.items():
            acc.setdefault(k, []).append(v)
    return acc


''' Store data as keys,values '''
grouped_data = collections.OrderedDict()
for k, g in grouped:
    grouped_data[k] = accumulate(g)


''' Print the very first k1. After this we only need to print k2 and update the idx '''
header_with_1stK1 = io.StringIO(data).read().split('\n')[0:2]
print('\n'.join(header_with_1stK1))

''' make an empty new_k2 value. This k2 value is updated and carried on base on match between vals from two different rows. '''
k2_new = ''

for n in range(2):
    if n > 0:
        break. # just to run the loop one time and to prevent resetting of k2_new values to ‘’

    ''' Now, read as keys, values pairs for two consecutive keys '''
    for (k1, v1), (k2, v2) in zip(grouped_data.items(), islice(grouped_data.items(), 1, None)):

        v1_vals = ''.join(v1['vals'])
        v2_vals = ''.join(v2['vals'])

        v1_list = list(v1_vals)
        v2_list = list(v2_vals)

        ''' to check if there is any matching element '''
        commons = [x for x in v1_list if x in v2_list]

        v2_pos = ''.join(v2['pos'])


        ''' start updating the idx values '''
        if k2_new == '':
            if len(commons) > 0:
                k2_new = k1
                print('\t'.join([v2_pos, k2_new, v2_vals]))

            else:
                k2_new = ''
                print('\t'.join([v2_pos, k2, v2_vals]))


        elif k2_new != '':
            if len(commons) > 0:
                k2_new = k2_new
                print('\t'.join([v2_pos, k2_new, v2_vals]))

            else:
                k2_new = ''
                print('\t'.join([v2_pos, k2, v2_vals]))


print('\nUpdated the idx values')

Answer 1

I may contribute by first eliminating the itertools and collections module.

import csv
import io

---

---

Set the data

    data = '''pos\tidx\tvals
    23\t4\tabc
    25\t7\tatg
    29\t8\tctb
    35\t1\txyz
    37\t2\tmno
    39\t3\tpqr
    41\t6\trtu
    45\t5\tlfg''';
print('INPUT:\n'+data);

---

---

Create a "set" of OrderedDictionary object based on the data, and then save it as a list. This is because data_As_dict is not subscriptable, relatively not flexible for work in iteration, and each dictionary in it will be erased after being reused.

data_As_Dict = csv.DictReader(io.StringIO(data), delimiter='\t');
list_Of_Dict = [i for i in data_As_Dict];

---

---

Now it is saved as a list, we can access each row. We can check the condition by simply checking v1+v2. For example, if v1='abc' and v2='atg', then v1+v2='abcatg'. We have a pattern, 'a' occurs twice, and the others are unique. So here is your condition : len(set(v1+v2)) != len(v1+v2)

Here is the code for creating the output,

for i in range(0, len(list_Of_Dict)-1):
    v1 = list_Of_Dict[i]['vals'];
    v2 = list_Of_Dict[i+1]['vals'];
    if len(set(v1+v2)) != len(v1+v2):
        list_Of_Dict[i+1]['idx'] = list_Of_Dict[i]['idx'];


output_data = 'pos\tidx\tvals\n';
for i in list_Of_Dict:
    output_data += i['pos']+'\t'+i['idx']+'\t'+i['vals']+'\n';
print('OUTPUT:\n'+output_data);

---

---

Full code:

import csv
import io

data = '''pos\tidx\tvals
23\t4\tabc
25\t7\tatg
29\t8\tctb
35\t1\txyz
37\t2\tmno
39\t3\tpqr
41\t6\trtu
45\t5\tlfg''';

print('INPUT:\n'+data);
data_As_Dict = csv.DictReader(io.StringIO(data), delimiter='\t');

list_Of_Dict = [i for i in data_As_Dict];

for i in range(0, len(list_Of_Dict)-1):
    v1 = list_Of_Dict[i]['vals'];
    v2 = list_Of_Dict[i+1]['vals'];
    if len(set(v1+v2)) != len(v1+v2):
        list_Of_Dict[i+1]['idx'] = list_Of_Dict[i]['idx'];


output_data = 'pos\tidx\tvals\n';
for i in list_Of_Dict:
    output_data += i['pos']+'\t'+i['idx']+'\t'+i['vals']+'\n';
print('OUTPUT:\n'+output_data);

---