A large csv I was given has a large table of flight data. A function I wrote to help parse it iterates over the column of Flight ID's, and then returns a dictionary containing the index and value of every unique Flight ID in order of first appearance.

Dictionary = { Index: FID, ... }

This comes as a quick adjustment to an older function that didn't require having to worry about FID repeats in the column (a few hundred thousand rows later...).

Right now, I have it iterating over and comparing each value in order. If a value is equal to the value after it, it skips it. If the next value is different, it stores the value in the dictionary. I changed it to now also check if that value has already occured before, and if so, to skip it. Here's my code:

def DiscoverEarliestIndex(self, number):                                             
    thegoodshit = {}                                                        
    columnvalues = self.column(number)                                             
    column_enum = {}                                                         
    for a, b in enumerate(columnvalues):                                           
        column_enum[a] = b                                                   
        i = 0                                                                                                                    
    while i < (len(columnvalues) - 1):                                             
        next = column_enum[i+1]                                              
        if columnvalues[i] == next:                                                
            i += 1                                                          
        else:                                                               
            if next in thegoodshit.values():                                
                i += 1                                                      
                continue                                                    
            else:                                                           
                thegoodshit[i+1]= next                                      
                i += 1                                                      
    else:                                                                   
        return thegoodshit 

It's very inefficient, and slows down as the dictionary grows. The column has 5.2 million rows, so it's obviously not a good idea to handle this much with Python, but I'm stuck with it for now.

Is there a more efficient way to write this function?