Fill missing dates into array of np.datetime

Question

Here is the problem:

Given an array of a non-continuous sequence of dates. Make it a continuous sequence of dates, by filling in the missing dates.

# Input

dates = np.arange(np.datetime64('2018-02-01'), np.datetime64('2018-02-25'), 2)

print(dates)
#> ['2018-02-01' '2018-02-03' '2018-02-05' '2018-02-07' '2018-02-09'
#>  '2018-02-11' '2018-02-13' '2018-02-15' '2018-02-17' '2018-02-19'
#>  '2018-02-21' '2018-02-23']

And here is my solution:

import numpy as n
dates = np.arange(np.datetime64('2018-02-01'), np.datetime64('2018-02-25'), 2)
stride = (dates[1] - dates[0])
result = np.arange(np.datetime64(dates[0]), np.datetime64(dates[-1] + stride))
print(dates)
print(result)

Is there a better way to do this task?

Answer 1

As you will be able to see from your provided sample input, your code does not produce the intended result. Here is a minimal example:

dates = np.arange(np.datetime64('2018-02-01'), np.datetime64('2018-02-05'), 2)
stride = (dates[1] - dates[0])
result = np.arange(np.datetime64(dates[0]), np.datetime64(dates[-1] + stride))

print(dates)   # > ['2018-02-01' '2018-02-03']
print(result)  # > ['2018-02-01' '2018-02-02' '2018-02-03' '2018-02-04']

result includes 2018-02-04, which is outside of the range of dates. This is only the simplest case for which this code fails, it will also fail for input arrays that are not sorted in ascending order. The only case your code would cover correctly is an input array of continuous dates (sorted in ascending order). There are two central errors in your code:

---

We do not need stride

We want the output range to include everything up until and including the maximum value of our input array. Since the stop argument to numpy.arange is exclusive, we need the following:

stop:
maximum of input array + 1

last element of output array:
(maximum of input array + 1) - 1
= maximum of input array

Your current implementation does the following:

stop:
maximum of input array + stride

last element of output array:
(maximum of input array + stride) - 1

The larger the difference between dates[0] and dates[1] (= stride), the more undesired values will be contained at the end of our output array.

---

The maximum is not always the last element

So we need to access the input array's maximum value to get the correct stop argument for numpy.arange. This will only be the last element if the input array is sorted in ascending order or if we get rather lucky, both cases are not specified by the problem statement.

---

Correcting these two errors will give us this simple implementation:

def make_continuous_sorted(np_array):
    return np.arange(np.min(np_array), np.max(np_array) + 1)