I have unstructured input-output data, f=f(x). I want to estimate the range of f as a function of x, given limited sampled. So, I tried to discretize the data use small bins in x to estimate the variance of f associated with that bin. After a lot of work, I cobbled together a working code. But, I'm sure there must be a better way. My results are very sensitive to the bin size. Please review my approach and let me know how it can be improved.

from scipy.stats import binned_statistic
import matplotlib.pyplot as plt
import numpy as np

# Choose the number of bins
NBINS = 100

# define custom function for findin range
def myrangefunciton(arrA):
    return(np.max(arrA) - np.min(arrA))

# set seed for reproducability, there is nothing special about this seed

# generate sample data
x = np.random.normal(3, 1, 1000)
f = np.random.normal(0, np.max([x, np.zeros(x.size)], 0))

# compute ranges of f across x
stats, binEdges, binNum = binned_statistic(x, f,
                                           statistic=myrangefunciton, bins=100)

# scatter data
plt.scatter(x, f, s=0.5, label='Observed f(x) Data', c='k')

# transform the bin edges to centered x values
rangeX = binEdges[:-1] + (binEdges[1] - binEdges[0]) / 2.

# plot ranges to envelop the mean, which is 0.
plt.plot(rangeX, stats / 2, label='Estimated Range', c='blue')
plt.plot(rangeX, -1 * stats / 2, c='blue')

# complete summary plot
plt.title("%i bins" % NBINS)
plt.legend(prop={'size': 4})

enter image description here

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.