Skip to main content
5 of 8
Adding another method using next instead of index
DevLounge
  • 208
  • 1
  • 7

So here is what I came up with, now that I had access to a computer:

raw_data.csv:

20110117559515,1,10,faa
20110117559515,2,20,bar
20110117559515,3,30,baz
20110117559572,4,40,fii
20110117559572,5,50,bir
20110117559572,6,60,biz
20110117559574,7,70,foo
20110117559587,8,80,bor
20110117559588,9,90,boz

code (written in several steps for readability):

import csv
from collections import defaultdict

if __name__ == '__main__':
    
    rows = [row for row in csv.reader(open('raw_data.csv', 'r'), delimiter=',')]
    cols = zip(*rows)
    unik = set(cols[0])
    
    positions = [cols[0].index(x) for x in unik]
    results   = [rows[p] for p in positions]
    indexes   = [rows.index(result) for result in results]

    indexed = defaultdict(list)
    
    for result in results:
        indexed[rows.index(result)] = result
    
    print 'positions:'
    pprint(positions)

    print '\nresults:'
    pprint(results)

    print '\nindexes:'
    pprint(indexes)

    print '\nindexed:'
    pprint(dict(indexed))

Output (Python counts from 0, not 1):

positions:
[3, 7, 0, 6, 8]

results:
[['20110117559572', '4', '40', 'fii'],
 ['20110117559587', '8', '80', 'bor'],
 ['20110117559515', '1', '10', 'faa'],
 ['20110117559574', '7', '70', 'foo'],
 ['20110117559588', '9', '90', 'boz']]

indexes:
[3, 7, 0, 6, 8]

indexed:
{0: ['20110117559515', '1', '10', 'faa'],
 3: ['20110117559572', '4', '40', 'fii'],
 6: ['20110117559574', '7', '70', 'foo'],
 7: ['20110117559587', '8', '80', 'bor'],
 8: ['20110117559588', '9', '90', 'boz']}

Simplified code:

import csv
from collections import defaultdict
from pprint import pprint

if __name__ == '__main__':
    
    rows = [row for row in csv.reader(open('raw_data.csv', 'r'), delimiter=',')]
    cols = zip(*rows)
    unik = set(cols[0])
    
    indexed = defaultdict(list)
    
    for x in unik:
        i = cols[0].index(x)
        indexed[i] = rows[i]
        
    print 'indexed:'
    pprint(dict(indexed))

Output:

indexed:
{0: ['20110117559515', '1', '10', 'faa'],
 3: ['20110117559572', '4', '40', 'fii'],
 6: ['20110117559574', '7', '70', 'foo'],
 7: ['20110117559587', '8', '80', 'bor'],
 8: ['20110117559588', '9', '90', 'boz']}

Second method:

rows = [row for row in csv.reader(open('raw_data.csv', 'r'), delimiter=',')]
    cols = zip(*rows)
    unik = set(cols[0])
    
    indexed = defaultdict(list)
    
    for x in unik:
        i = next(index for index,fid in enumerate(cols[0]) if fid == x)
        indexed[i] = rows[i]

Is this better in terms of memory usage or performance?

DevLounge
  • 208
  • 1
  • 7