# Update the idx values if vals element match between consecutive rows

I want to update the numbers in idx field if there is any matching letter between the vals from two consecutive row.

Input data = '''pos\tidx\tvals
23\t4\tabc
25\t7\tatg
29\t8\tctb
35\t1\txyz
37\t2\tmno
39\t3\tpqr
41\t6\trtu
45\t5\tlfg'''


Explantion: Since there is letteramatching between idx 4 and 7 the idx at pos 25 will be updated to 4, but again there istmatching between vals at pos 25 and 29 we update the idx at 29 also to 4 instead of just 7.

#Expected output to a file (tab separated):
pos    idx    vals
23    4    abc
25    4    atg
29    4    ctb
35    1    xyz
37    2    mno
39    3    pqr
41    3    rtu
45    5    lfg


I have written the given workable code (below) so far, and would also like to

• write the expected output to a file
• optimize the code for the work I am doing.
• the answer has to follow my method of reading two consecutive rows (as keys, values) pairs at a time in ordered way. The reason is this question is just a trial of other problem I am trying to solve. Other parts of the code can be optimized in any pythonic way.

Code:

import csv
import itertools
import collections
import io
from itertools import islice

data = '''pos\tidx\tvals
23\t4\tabc
25\t7\tatg
29\t8\tctb
35\t1\txyz
37\t2\tmno
39\t3\tpqr
41\t6\trtu
45\t5\tlfg'''

grouped = itertools.groupby(data_As_Dict, key=lambda x: x['idx'])

''' Function to read the data as key, val pairs in Ordered way.'''
def accumulate(data):
acc = collections.OrderedDict()
for d in data:
for k, v in d.items():
acc.setdefault(k, []).append(v)
return acc

''' Store data as keys,values '''
grouped_data = collections.OrderedDict()
for k, g in grouped:
grouped_data[k] = accumulate(g)

''' Print the very first k1. After this we only need to print k2 and update the idx '''

''' make an empty new_k2 value. This k2 value is updated and carried on base on match between vals from two different rows. '''
k2_new = ''

for n in range(2):
if n > 0:
break. # just to run the loop one time and to prevent resetting of k2_new values to ‘’

''' Now, read as keys, values pairs for two consecutive keys '''
for (k1, v1), (k2, v2) in zip(grouped_data.items(), islice(grouped_data.items(), 1, None)):

v1_vals = ''.join(v1['vals'])
v2_vals = ''.join(v2['vals'])

v1_list = list(v1_vals)
v2_list = list(v2_vals)

''' to check if there is any matching element '''
commons = [x for x in v1_list if x in v2_list]

v2_pos = ''.join(v2['pos'])

''' start updating the idx values '''
if k2_new == '':
if len(commons) > 0:
k2_new = k1
print('\t'.join([v2_pos, k2_new, v2_vals]))

else:
k2_new = ''
print('\t'.join([v2_pos, k2, v2_vals]))

elif k2_new != '':
if len(commons) > 0:
k2_new = k2_new
print('\t'.join([v2_pos, k2_new, v2_vals]))

else:
k2_new = ''
print('\t'.join([v2_pos, k2, v2_vals]))

print('\nUpdated the idx values')


I may contribute by first eliminating the itertools and collections module.

import csv
import io


Set the data

    data = '''pos\tidx\tvals
23\t4\tabc
25\t7\tatg
29\t8\tctb
35\t1\txyz
37\t2\tmno
39\t3\tpqr
41\t6\trtu
45\t5\tlfg''';
print('INPUT:\n'+data);


Create a "set" of OrderedDictionary object based on the data, and then save it as a list. This is because data_As_dict is not subscriptable, relatively not flexible for work in iteration, and each dictionary in it will be erased after being reused.

data_As_Dict = csv.DictReader(io.StringIO(data), delimiter='\t');
list_Of_Dict = [i for i in data_As_Dict];


Now it is saved as a list, we can access each row. We can check the condition by simply checking v1+v2. For example, if v1='abc' and v2='atg', then v1+v2='abcatg'. We have a pattern, 'a' occurs twice, and the others are unique. So here is your condition : len(set(v1+v2)) != len(v1+v2)

Here is the code for creating the output,

for i in range(0, len(list_Of_Dict)-1):
v1 = list_Of_Dict[i]['vals'];
v2 = list_Of_Dict[i+1]['vals'];
if len(set(v1+v2)) != len(v1+v2):
list_Of_Dict[i+1]['idx'] = list_Of_Dict[i]['idx'];

output_data = 'pos\tidx\tvals\n';
for i in list_Of_Dict:
output_data += i['pos']+'\t'+i['idx']+'\t'+i['vals']+'\n';
print('OUTPUT:\n'+output_data);


Full code:

import csv
import io

data = '''pos\tidx\tvals
23\t4\tabc
25\t7\tatg
29\t8\tctb
35\t1\txyz
37\t2\tmno
39\t3\tpqr
41\t6\trtu
45\t5\tlfg''';

print('INPUT:\n'+data);

list_Of_Dict = [i for i in data_As_Dict];

for i in range(0, len(list_Of_Dict)-1):
v1 = list_Of_Dict[i]['vals'];
v2 = list_Of_Dict[i+1]['vals'];
if len(set(v1+v2)) != len(v1+v2):
list_Of_Dict[i+1]['idx'] = list_Of_Dict[i]['idx'];

output_data = 'pos\tidx\tvals\n';
for i in list_Of_Dict:
output_data += i['pos']+'\t'+i['idx']+'\t'+i['vals']+'\n';
print('OUTPUT:\n'+output_data);