1
\$\begingroup\$

Context: My question comes from needing to iterate different np.arange arrays. The main motivation is to calculate all the possible combinations between the arrays, let's say a, b, c, x, y, z where the last 3 (x, y, z) can be arrays OR floats.

The reason behind is that every combination can be evaluated then in equations. Floats OR values are considered because x, y, z represent parameters (that can be fixed or considered to have errors) and a, b, c are variables. This is useful when wanting to evaluate a set of differential equations solutions. In a mathematical context some would say "the raison d'être is to explore the space of parameters".

To calculate every possible combination I know I can iterate using for-loops. The problem arises when considering the exceptions: as x, y, z can be lists "L" or numbers "N", I find the cases:

#Different combinations:

options=[['N',1],['L',2]]
for i in options:
    for j in options:
        for k in options:
            print(i[0]+j[0]+k[0]+' w/ total combinations: '+str(2*2*2*i[1]*j[1]*k[1]))

which are NNN, NNL, NLL, NLN, LNL, LLN, LNN, LLL. I've also calculated the length of the resulting possibilities, for the case of len()=2 arrays.

What is the code doing?: calculating every possibility. If there's a number instead of a list (or array) it's not iterated but appended constantly through every possibility for the other variables that do "change".

But the problem is: considering each case separatedly is unefficent and messy code. Question: how could I do it in a more general and compacted way?

You can check the code here:

import random
import numpy as np
import numbers

a = np.random.uniform(1,10,2)
b = np.random.uniform(1,10,2)
c = np.random.uniform(1,10,2)

w=[36] #this is a len(w)=1 list only to manage exceptions, but is equivalent to have a number instead of a list/array.

x = np.random.normal(2,1e-1,2) #This could be a float
y = np.random.normal(3,1e-1,2) #This could be a float
z = np.random.normal(4,1e-1,2) #This could be a float


#Now interate

def iter_func(args):

    returned_array = [] #this is the output list-of-lists with all the combinations.

    a,b,c,x,y,z = args #numbers or arrays; variables or parameters (w/ or w/o errors in a mathematical context)
    
    for avals in a:
        for bvals in b:
            for cvals in c:
            
                if isinstance(x,np.ndarray)==True:                
                    
                    for xvals in x:
                        
                        if isinstance(y,np.ndarray)==True:
                        
                            for yvals in y:
                                
                                if isinstance(z,np.ndarray)==True:
                        
                                    for zvals in z:
                                        returned_array.append([avals,bvals,cvals,xvals,yvals,zvals])
                                
                                elif isinstance(z,numbers.Real)==True or isinstance(z,list)==True and len(z)==1:
                                        returned_array.append([avals,bvals,cvals,xvals,yvals,z])
                        
                        elif isinstance(y,numbers.Real)==True or isinstance(y,list)==True and len(y)==1:
                            returned_array.append([avals,bvals,cvals,xvals,y,z])
                            
                elif isinstance(x,numbers.Real)==True or isinstance(x,list)==True and len(x)==1:
                    
                    if isinstance(y,np.ndarray)==True:
                       
                            for yvals in y:
                                
                                if isinstance(z,np.ndarray)==True:
                        
                                    for zvals in z:
                                        returned_array.append([avals,bvals,cvals,x,yvals,zvals])
                                
                                elif isinstance(z,numbers.Real)==True or isinstance(z,list)==True and len(z)==1:
                                        returned_array.append([avals,bvals,cvals,x,yvals,z])
                        
                    elif isinstance(y,numbers.Real)==True or isinstance(y,list)==True and len(y)==1:
                        
                        if isinstance(z,np.ndarray)==True:
                        
                            for zvals in z:
                                returned_array.append([avals,bvals,cvals,x,y,zvals])
                                
                        elif isinstance(z,numbers.Real)==True or isinstance(z,list)==True and len(z)==1:
                            returned_array.append([avals,bvals,cvals,x,y,z])
                        
    return returned_array

#Now I'm testing different possibilities

#Case test: last 3 are lists of numbers
returned_array = iter_func([a,b,c,w,w,w])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 1: NNN
returned_array = iter_func([a,b,c,w[0],w[0],w[0]])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 2: NNL
returned_array = iter_func([a,b,c,w[0],w[0],z])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 3: NLN 
returned_array = iter_func([a,b,c,w[0],y,w[0]])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 4: NLL   
returned_array = iter_func([a,b,c,w[0],y,z])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 5: LNN    
returned_array = iter_func([a,b,c,x,w[0],w[0]])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 6: LNL    
returned_array = iter_func([a,b,c,x,w[0],z])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 7: LLN    
returned_array = iter_func([a,b,c,x,y,w[0]])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

#Case 8: LLL   
returned_array = iter_func([a,b,c,x,y,z])
print('total combinations: '+str(len(returned_array)))
for i in returned_array:
    print(i)
print('\n')

with output:

total combinations: 8
[2.8336495994988162, 5.295916749347686, 4.929950792386235, [36], [36], [36]]
[2.8336495994988162, 5.295916749347686, 9.493663046965622, [36], [36], [36]]
[2.8336495994988162, 9.869611330430935, 4.929950792386235, [36], [36], [36]]
[2.8336495994988162, 9.869611330430935, 9.493663046965622, [36], [36], [36]]
[2.741768057298594, 5.295916749347686, 4.929950792386235, [36], [36], [36]]
[2.741768057298594, 5.295916749347686, 9.493663046965622, [36], [36], [36]]
[2.741768057298594, 9.869611330430935, 4.929950792386235, [36], [36], [36]]
[2.741768057298594, 9.869611330430935, 9.493663046965622, [36], [36], [36]]


total combinations: 8
[2.8336495994988162, 5.295916749347686, 4.929950792386235, 36, 36, 36]
[2.8336495994988162, 5.295916749347686, 9.493663046965622, 36, 36, 36]
[2.8336495994988162, 9.869611330430935, 4.929950792386235, 36, 36, 36]
[2.8336495994988162, 9.869611330430935, 9.493663046965622, 36, 36, 36]
[2.741768057298594, 5.295916749347686, 4.929950792386235, 36, 36, 36]
[2.741768057298594, 5.295916749347686, 9.493663046965622, 36, 36, 36]
[2.741768057298594, 9.869611330430935, 4.929950792386235, 36, 36, 36]
[2.741768057298594, 9.869611330430935, 9.493663046965622, 36, 36, 36]


total combinations: 16
[2.8336495994988162, 5.295916749347686, 4.929950792386235, 36, 36, 3.8501788450044114]
[2.8336495994988162, 5.295916749347686, 4.929950792386235, 36, 36, 3.895647492990764]
[2.8336495994988162, 5.295916749347686, 9.493663046965622, 36, 36, 3.8501788450044114]
[2.8336495994988162, 5.295916749347686, 9.493663046965622, 36, 36, 3.895647492990764]
[2.8336495994988162, 9.869611330430935, 4.929950792386235, 36, 36, 3.8501788450044114]
[2.8336495994988162, 9.869611330430935, 4.929950792386235, 36, 36, 3.895647492990764]
[2.8336495994988162, 9.869611330430935, 9.493663046965622, 36, 36, 3.8501788450044114]
[2.8336495994988162, 9.869611330430935, 9.493663046965622, 36, 36, 3.895647492990764]
[2.741768057298594, 5.295916749347686, 4.929950792386235, 36, 36, 3.8501788450044114]
[2.741768057298594, 5.295916749347686, 4.929950792386235, 36, 36, 3.895647492990764]
[2.741768057298594, 5.295916749347686, 9.493663046965622, 36, 36, 3.8501788450044114]
[2.741768057298594, 5.295916749347686, 9.493663046965622, 36, 36, 3.895647492990764]
[2.741768057298594, 9.869611330430935, 4.929950792386235, 36, 36, 3.8501788450044114]
[2.741768057298594, 9.869611330430935, 4.929950792386235, 36, 36, 3.895647492990764]
[2.741768057298594, 9.869611330430935, 9.493663046965622, 36, 36, 3.8501788450044114]
[2.741768057298594, 9.869611330430935, 9.493663046965622, 36, 36, 3.895647492990764]

Q: is this called "variable-depth for-loops", "nested for-loops"? Any comment on this would be highly appreciated.

I'm not very familiar with Python but Q: I think it could be done in a simplier way. What would you suggest? Any hint? Maybe a way to do it with pandas, numpy or in a big list comprehension? (I do not really know if possible). I apologize if the question is vague. I'd be happy if you suggest anything to make the question clearer.

Improve this question
\$\endgroup\$
0

1 Answer 1

Reset to default
4
\$\begingroup\$

Your question and your code are both fairly confused, but let's attempt to dig into it anyway.

The return type of your function is a big, crazy mix of Python native sequences and Numpy arrays, when it should be a single two-dimensional Numpy array.

iter_func does a poor job of explaining what this is supposed to accomplish, but I'm leaving it for now.

Delete all of your isinstance checks and replace them with a type-coerce via np.atleast_1d.

Don't write your code to require six arguments; accept an arbitrary number of arguments.

Delete basically all of your code. Recognise that this is a Cartesian product, and whereas Numpy doesn't have a built-in for this, this is a solved problem on the internet.

Add PEP484 type hints, write a wrapper for the above SO answer that accommodates your lists, scalars and arrays, and we have

from typing import Union, Sequence

import numpy as np
from numbers import Real


def cartesian_product(*arrays: np.ndarray) -> np.ndarray:
    """
    See
    https://stackoverflow.com/questions/11144513/cartesian-product-of-x-and-y-array-points-into-single-array-of-2d-points
    """
    la = len(arrays)
    dtype = np.result_type(*arrays)
    arr = np.empty([len(a) for a in arrays] + [la], dtype=dtype)
    for i, a in enumerate(np.ix_(*arrays)):
        arr[..., i] = a
    return arr.reshape(-1, la)


def iter_func(
    *args: Union[Real, Sequence[Real], np.ndarray],
) -> np.ndarray:
    return cartesian_product(*(
        np.atleast_1d(a) for a in args
    ))
Improve this answer
\$\endgroup\$
4
  • \$\begingroup\$ Thanks much for the attempt and effort! This works exactly as I expected and allows me to study about Cartesian products. I've redacted the question again aiming for a better explanation. Also, I think that I could have converted numbers into constant arrays and use for loops with a break (to iterate only 1 time) if the 1st element of a constant array is equivalent to the 2nd . This might be simplier than my first attempt. Should I forget this? What do you think? A cartesian product is the standard? \$\endgroup\$
    – nuwe
    Dec 5, 2021 at 15:46
  • 1
    \$\begingroup\$ Should I forget this? Yeah, probably. for-loops are generally inappropriate to use with Numpy arrays when vectorised solutions exist. The loop still exists, but it's done for you in a compiled library that runs much faster. \$\endgroup\$
    – Reinderien
    Dec 5, 2021 at 16:22
  • \$\begingroup\$ Just asking for more information: if I need to impose a condition over every combination should I write it indented inside for i, a in enumerate(np.ix_(*arrays))? e.g., if sum() > 20 discard the list, else append the list+result of sum(). Again, I could iterate over possible combinations but I suspect this is not the fastest way. Should I open a new question? \$\endgroup\$
    – nuwe
    Dec 5, 2021 at 16:36
  • 1
    \$\begingroup\$ should I write it indented inside [a for loop]? No. You should invest some research time into how Numpy uses vectorised logic and slicing. Should I open a new question? Yes; though if it's "how do I filter in Numpy", it would belong on StackOverflow and not CodeReview. \$\endgroup\$
    – Reinderien
    Dec 5, 2021 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.