# Is there a way to make reshaping arrays shorter and cleaner?

I'm trying to reshape my input arrays in a way that's described below to be able to fit a curve on the data points, plot it, etc. It works fine, but I'm afraid it's not the most efficient way of doing it, and also it's hard to understand and read. It's part of a bigger function, but I only post here the critical part and not the whole function. I also added little comments to help you understand what's going on. I know it's ugly.

import numpy as np
from itertools import chain

#example inputs
delay = np.array([10,20,30,40,50,60,70,None])
omega = [[1,2,3,4],[1.1,2.2,3.3,None],[1.4,2.4,3.4,None],[1.5,2.8,None,None],
[1.8,2.9,None,None],[1.9,None,None,None],[2.0,None,None,None],[None,None,None,None]]

"""
desired output explained:
with delay's first component 10 I want to assign 1, 2, 3 and 4
so the outputs starts like this: [10,10,10,10] , [1,2,3,4]
then with delay's second component 20 I want to assign 1.1, 2.2, 3.3 and drop the None value.
at the moment the outputs should look like this: [10,10,10,10,20,20,20] , [1,2,3,4,1.1,2.2,3.3]
and so on.
"""

def spp_method(omegas, delays):

#dropping all None values from delays
delays = delays[delays != np.array(None)]
#dropping all the None values from omegas
omegas_ext = []
for element in omegas:
item = [x for x in element if x is not None]
omegas_ext.append(item)

#expand delays with the number of values in omegas_ext appropriate component
delays_exp = []
for idx in range(len(omegas_ext)):
if len(omegas_ext[idx])>1:
value = delays[idx]
elif len(omegas_ext[idx]) == 1:
value = delays[idx]
delays_exp.append([value])
#put the values into simple array to plot and to fit curve.
delays_unpacked = []
omegas_unpacked = []
for element in omegas_ext:
for item in element:
omegas_unpacked.append(item)
delays_unpacked = list(chain.from_iterable(delays_exp))
return np.array(delays_unpacked), np.array(omegas_unpacked)

y, x = spp_method(omega, delay)

print(x)
#outputs to: [1.  2.  3.  4.  1.1 2.2 3.3 1.4 2.4 3.4 1.5 2.8 1.8 2.9 1.9 2. ]
print(y)
#outputs to: [10 10 10 10 20 20 20 30 30 30 40 40 50 50 60 70]



which are correct.

Any improvements in the code are well-appreciated.

• Welcome to Code Review. Please change your title so that it describes what you are trying to accomplish, not what you expect from the review (see also How to Ask). Aug 29 '19 at 11:18

You took a curved path to where you want to go :)

It seems you are ill at ease at building flat lists and take long routes at building nested ones then unpacking them. But you can replace :

    omegas_ext = []
for element in omegas:
item = [x for x in element if x is not None]
omegas_ext.append(item)
# [...]
for element in omegas_ext:
for item in element:
omegas_unpacked.append(item)


By:

    omegas_ext = []
for element in omegas:
item = [x for x in element if x is not None]
omegas_ext.extend(item)


Second, this if else is unecessary. [value] * 1 is equivalent to [value].

        if len(omegas_ext[idx])>1:
value = delays[idx]
elif len(omegas_ext[idx]) == 1:
value = delays[idx]
delays_exp.append([value])


Can be replaced by:

    if len(omegas_ext[idx])>=1:
delays_exp.append([delays[idx]]*(len(omegas_ext[idx])))


Again you unpack this later on. So you could extend and this would also make it unecessary to have this check for an element (since extending with an empty list is a no-op)

    delays_exp.extend([delays[idx]*len(omegas_ext[idx]))
# itertools.chain no longer needed


Finally, you missed the fact you can build the two lists in a single for, by using zip. This would save you the need to use an index to recompute the length of the omegas_ext items.

Here would be the function :

def spp_method(omegas, delays):
delays = delays[delays != np.array(None)]
omegas_ext = []
delays_exp = []
for delay, element in zip(delays, omegas):
item = [x for x in element if x is not None]
omegas_ext.extend(item)
delays_exp.extend(len(item) * [delay])
return np.array(delays_exp), np.array(omegas_ext)


Code returns the same output.

• Thank you for the answer, it is much better like this. Aug 29 '19 at 11:29
In [15]: delay = np.array([10,20,30,40,50,60,70,None])
...: omega = [[1,2,3,4],[1.1,2.2,3.3,None],[1.4,2.4,3.4,None],[1.5,2.8,None,None],
...:          [1.8,2.9,None,None],[1.9,None,None,None],[2.0,None,None,None],[None,None,None,None]]
...:


We can use a list comprehension to create a list of tuples. One tuple for each non-None element in omega.

In [16]: [(i,j) for i,k in zip(delay, omega) for j in k if j is not None]
Out[16]:
[(10, 1),
(10, 2),
(10, 3),
(10, 4),
(20, 1.1),
(20, 2.2),
(20, 3.3),
(30, 1.4),
(30, 2.4),
(30, 3.4),
(40, 1.5),
(40, 2.8),
(50, 1.8),
(50, 2.9),
(60, 1.9),
(70, 2.0)]


Then use the zip(*) idiom to 'transpose' this into two tuples:

In [17]: x,y = zip(*_)
In [18]: x
Out[18]: (10, 10, 10, 10, 20, 20, 20, 30, 30, 30, 40, 40, 50, 50, 60, 70)
In [19]: y
Out[19]: (1, 2, 3, 4, 1.1, 2.2, 3.3, 1.4, 2.4, 3.4, 1.5, 2.8, 1.8, 2.9, 1.9, 2.0)


A numpy version:

In [40]: A = delay.repeat(4).reshape(8,4)
In [41]: B = np.array(omega)
In [42]: mask = np.frompyfunc(lambda x: x is not None,1,1)(_21).astype(bool)
Out[43]:
array([10, 10, 10, 10, 20, 20, 20, 30, 30, 30, 40, 40, 50, 50, 60, 70],
dtype=object)
Out[44]:
array([1, 2, 3, 4, 1.1, 2.2, 3.3, 1.4, 2.4, 3.4, 1.5, 2.8, 1.8, 2.9, 1.9,
2.0], dtype=object)


If omega had np.nan instead of None I could have made the mask without the frompyfunc iteration.

The arrays:

In [45]: A
Out[45]:
array([[10, 10, 10, 10],
[20, 20, 20, 20],
[30, 30, 30, 30],
[40, 40, 40, 40],
[50, 50, 50, 50],
[60, 60, 60, 60],
[70, 70, 70, 70],
[None, None, None, None]], dtype=object)
In [46]: B
Out[46]:
array([[1, 2, 3, 4],
[1.1, 2.2, 3.3, None],
[1.4, 2.4, 3.4, None],
[1.5, 2.8, None, None],
[1.8, 2.9, None, None],
[1.9, None, None, None],
[2.0, None, None, None],
[None, None, None, None]], dtype=object)