2
\$\begingroup\$

I want to return a NumPy array of Element IDs with at least one Node being in the interesting Nodes list.

I have:

  • A list of interesting Nodes, Interesting_Nodes.
    The size is ~300,000 items.

  • A list of sublists which represent Node forming an Element, Nodes.
    The size is ~1,000,000 lists of ~20 values.

    Nodes[i] contain the 20 Nodes creating Elements[i]. Which is an integer ID.

  • A list of Elements, Elements. The size is ~1,000,000 items.

Here is an example of these lists.

import numpy as np
Interesting_Nodes=[1,2,10,40,400,1000]
Elements=[1,2,3]
Nodes=[[1,20,25],[30,400,35],[500,501,502]]

In this case the function will return [1,2] because Elements[0] have the Node 1 in the Interesting_Nodes list and Elements[1] contains the Node 400 which is also in the Interesting_Nodes list

I wrote this it seems to work but is very slow. Is there a way to perform this faster?

def recuperation_liste_element_dans_domaine_interet(Interesting_Nodes,Elements,Nodes):
    Liste_elements=list([])
    for n in Interesting_Nodes:
        id=np.where(Nodes==n)
        Liste_elements=Liste_elements+list(Elements[id[0]])
    nbrListe = list(set(Liste_elements)) # remove all the element duplication
    return np.array(nbrListe) 

Another way to perform that I think could be: (be still too slow)

def recuperation_liste_element_dans_domaine_interet(Interesting_Nodes,Elements,Nodes):
    Liste_elements=[0]*len(Elements)
    compteur=0
    j=0
    for n in Nodes:
        if any(i in Interesting_Nodes for i in n):
            Liste_elements[compteur]=Elements[j]
            compteur=compteur+1
        j=j+1
    Liste_elements=Liste_elements[:compteur]  
    return np.array(Liste_elements)
\$\endgroup\$
0
3
\$\begingroup\$

I'm assuming Elements is always a list of continuous indices from zero to the length of Nodes minus one and thus strictly not needed.

As a general guideline, explicit loops tend to be slow and also hard to read (and non-Pythonic).

To solve your problem efficiently, use data structures (namely sets):

import numpy as np

# Suppose you start with these lists
u = [1,2,10,40,400,1000]
n = [[1,20,25],[30,400,35],[500,501,502]]

# Convert them into sets
u = set(u)
n = [set(x) for x in n]

#  An individual bit is true if the corresponding list is "interesting"
idx = [bool(a.intersection(u)) for a in n]

# Perhaps convert to integer indices if you wish
np.where(idx)
\$\endgroup\$
4
  • \$\begingroup\$ thanks a lot for your help. I figured out myself that set is much more powerful to do what I expected 1 hour ago ;) But not as "clean" as you write. I think it would do the job perfectly ! Thanks again ! \$\endgroup\$
    – Bnj_07
    Oct 2 '20 at 18:27
  • \$\begingroup\$ By the way in fact i need Elements because it is not a continuous list. It coub be [10,1000,300]. But with idx I suppose I can juste write Elements[idx] to have my answer. \$\endgroup\$
    – Bnj_07
    Oct 2 '20 at 18:45
  • \$\begingroup\$ Did you see an improvement in your "very slow" execution time? What kind of improvement did you see? \$\endgroup\$
    – rajah9
    Oct 3 '20 at 13:29
  • 1
    \$\begingroup\$ a HUGE improvement !!! With the previous version, with the "utltimate" amount of data, i never manage to reach the end. With the new version, the code finish in few minutes. Thank you again @Juho \$\endgroup\$
    – Bnj_07
    Oct 4 '20 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.