I have unstructured input-output data, f=f(x). I want to estimate the range of f as a function of x, given limited sampled. So, I tried to discretize the data use small bins in x to estimate the variance of f associated with that bin. After a lot of work, I cobbled together a working code. But, I'm sure there must be a better way. My results are very sensitive to the bin size. Please review my approach and let me know how it can be improved.

from scipy.stats import binned_statistic
import matplotlib.pyplot as plt
import numpy as np

# Choose the number of bins
NBINS = 100

# define custom function for findin range
def myrangefunciton(arrA):
    return(np.max(arrA) - np.min(arrA))

# set seed for reproducability, there is nothing special about this seed

# generate sample data
x = np.random.normal(3, 1, 1000)
f = np.random.normal(0, np.max([x, np.zeros(x.size)], 0))

# compute ranges of f across x
stats, binEdges, binNum = binned_statistic(x, f,
                                           statistic=myrangefunciton, bins=100)

# scatter data
plt.scatter(x, f, s=0.5, label='Observed f(x) Data', c='k')

# transform the bin edges to centered x values
rangeX = binEdges[:-1] + (binEdges[1] - binEdges[0]) / 2.

# plot ranges to envelop the mean, which is 0.
plt.plot(rangeX, stats / 2, label='Estimated Range', c='blue')
plt.plot(rangeX, -1 * stats / 2, c='blue')

# complete summary plot
plt.title("%i bins" % NBINS)
plt.legend(prop={'size': 4})

enter image description here

enter image description here

| improve this question | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.