Python Genetic Algorithm Implementation

Question

I've made this to pick the LQR parameters for my self balancing robots simulation. So to break down what I have done,

Randomly Create Population

1. I create a random population. The population is a list of lists having 4 parameters each of which is randomly generated. But I have applied bounds for them using the values I got from my manual tuning for my simulation.

    def population_create(p_size,geno_size,bounds):
   
    p = [ random_param(geno_size,bounds) for i in range(p_size)]
    return p
   
   
   
   def random_param(g,b):
        return [random.uniform(b[i][0],b[i][1]) for i in range(g)]
   

Try it on Robot and Evaluate

2.Then I evaluate each parameter to see how much the wheels have rotated and how much the robot is inclined after one minute.

fitness = encoder_readings + tilt

So the closer the fitness is to zero the better the robot balances. This is done for the whole population

Create next generation

3.Then I make the next generation with Mutation, Crossover and passing healthy individuals directly.

def population_reproduce(p,fitness):

    size_p = len(p)
    new_p = []
    
    dataframe = pd.DataFrame({"Param":p,"Fitness":fitness})
    dataframe = dataframe.sort_values(['Fitness'])
    dataframe = dataframe.reset_index(drop=True)    

    sorted_p = dataframe['Param'].tolist()

    elite_part = round(ELITE_PART*size_p)
    new_p = new_p + sorted_p[:elite_part]

    for i in range(size_p-elite_part):
        mom = p[random.randint(0,size_p-1)]
        dad = p[random.randint(0,size_p-1)]
        child = crossover(mom,dad)
        child = mutate(child)
        new_p.append(child)
    
    return new_p

def crossover(p1,p2):
    
    crossover = []
    locii = [random.randint(0,8) for _ in range(len(p1))]
    
    for i in range(len(p1)):
        if locii[i]>4:
            crossover.append(p2[i])
        else:
            crossover.append(p1[i])
        
    return crossover

def mutate(c):
    size = len(c)
    for i in range(size):
        if random.random()< MUTATION_PROBABILITY:
            c[i] += random.gauss(c[i]/50,MUTATION_DEVIATION)
            
    return c   

This continues for some generations.

These are the links to the full codes.

Genetic Algorithm Supervisor

Population Script

Robot Controller

I would very much appreciate if you could take a look and let me know if this is a correct implementation of a genetic algorithm.

One observation I made is that the fitness doesn't converge to a lower value after each iteration. It randomly goes up and down. Could this be because of some inconsistency in my stimulation on Webots or is it a mistake in my code thats causing this?

Answer 1

On the style of the code:

While I'm not saying the code is or isn't correct, there's some small tweaks you could make it to improve it's legibility:

Spacing

Write code like you would write English (in some sense :P). After commas you needs spaces, so do

• def crossover(p1, p2): instead of def crossover(p1,p2):;
• random.uniform(b[i][0], b[i][1]) instead of random.uniform(b[i][0],b[i][1])
• same thing around binary ops: if locii[i] > 4: instead of if locii[i]>4:
• etc.

List comprehensions often do not need a leading space, so you can go with

p = [random_param(geno_size, bounds) for i in range(p_size)]

instead of

p = [ random_param(geno_size,bounds) for i in range(p_size)]

crossover

You have written:

def crossover(p1,p2):
    
    crossover = []
    locii = [random.randint(0,8) for _ in range(len(p1))]
    
    for i in range(len(p1)):
        if locii[i]>4:
            crossover.append(p2[i])
        else:
            crossover.append(p1[i])
        
    return crossover

Consider this alternative definition:

def crossover(p1, p2):
    
    crossover = []
    for p1_elem, p2_elem in zip(p1, p2):
        if random.randint(0,8) > 4:
            crossover.append(p2_elem)
        else:
            crossover.append(p1_elem)
        
    return crossover

Notice I don't precompute all locii, although you could do that, of course. Also note the usage of zip which means I can traverse both p1 and p2, instead of having to use indices to later grab the values. (You can read more about zip and find other tips and tricks here.

random_param

In line with one of the things I showed above, instead of doing

def random_param(g,b):
     return [random.uniform(b[i][0],b[i][1]) for i in range(g)]

you could iterate directly over b:

def random_param(g, b):
     return [random.uniform(bs[0], bs[1]) for bs in b]

(More on better iterating in Python here.)

Then, take that and unpack bs with * instead of with indices:

def random_param(g, b):
     return [random.uniform(*bounds) for bounds in b]