# Average across direction-specific average speeds and probabilities

I have a wind rose with 72 evenly-spaced directions, spanning 360°, each describing a direction-specific average wind speed and associated probability. We must condense this information to 36 evenly-spaced direction bins. "probability" is the normalized frequency of occurrences. The probabilities should sum to one. If 5° has a 0.1 probability. that means that the wind came from that direction 10% of the time.

I average across the midpoints of each bin, which results in undesirably shifting the wind rose so that the center is at 2.5 degrees. Is there a more elegant way to do this in theory or implementation?

import pandas as pd
import numpy as np
import wget

url = 'https://raw.githubusercontent.com/scls19fr/windrose/master/samples/amalia_directionally_averaged_speeds.txt'
dat.columns = ['direction', 'average_speed', 'probability']

# two steps
#   1. average frequency across speeds
#   2. compute average speed weighted by frequencies
#     (the scheme is to average across both midpoints. this
#      shift the centers by 2.5 degrees and bin by 10)
directions, speeds, frequencies = [], [], []
for i in range(dat.shape / 2):
directions.append(dat.direction[i * 2] + 2.5)
frequencies.append(np.mean(dat.probability[[i * 2, i * 2 + 1]].values))
speeds.append((dat.direction[i * 2] * dat.average_speed[i*2] +
dat.direction[i*2 + 1] * dat.average_speed[i * 2 + 1]) /
(dat.direction[i * 2] + dat.direction[i * 2+1]))

# save data
done = pd.DataFrame({'direction': directions,
'speed': speeds, 'probability': frequencies})
done.to_csv('wind_rose.csv')

• Also, there is a typo. It should be dat = pd.read_csv('./amalia_directionally_averaged_speeds.txt', sep=r'\s+'). – Graipher Mar 31 '17 at 9:48

The biggest issue with this code is it iterates over a DataFrame. Iterating over a DataFrame negates any performance improvements gained by using pandas. As much as possible, use operations that apply to the entire Series or an entire DataFrame at once. Allow the C code to do the looping for you so the data doesn't get pulled into slow Python objects.

There is no reason to download the data into a file using wget. This data can be downloaded directly into memory, and pandas supports http urls.

Look at where you place comments before blocks of code. These are perfect places to split up your code into functions.

"""Process unweighted wind data to file."""
import pandas as pd

URL = ('https://raw.githubusercontent.com/scls19fr/windrose/master/samples/'
'amalia_directionally_averaged_speeds.txt')
OUTPUT_PATH = 'wind_rose.csv'

"""Load wind data from url into a dataframe.

Args:
data_url(str|io):
A URL, path, or buffer containing wind data.

Returns:
A dataframe containing wind data
"""

names=['direction', 'average_speed', 'probability'])
return dat


You can use the names argument of read_csv instead of setting columns. wget is not used and the url is passed into read_csv since read_csv supports http.

def average_wind_data(dat):
"""Average and weight wind data.

Args:
dat(pd.DataFrame):
A dataframe containing unweighted wind data.
Returns:
A dataframe containing weighted, averaged wind data.

1. average frequency across speeds
2. compute average speed weighted by frequencies
(the scheme is to average across both midpoints. this
shift the centers by 2.5 degrees and bin by 10)
"""

avg_df = pd.DataFrame()
# shift(-1) leaves the NaN value at the end of the dataframe
avg_df['direction'] = (dat['direction'][::2] + 2.5).values
unweighted_speed = dat['direction'] * dat['average_speed']
return avg_df


Use shift() on Series or DataFrame objects to do vector operations on adjacent data. Use slice notation to get even data points ([::2] means all the entire index stepped over by 2). Do vector operations on entire series instead of each item on the series. This section of code is about 6x faster than the original iteration code.

def main(input_url, output_path):
"""Load wind data at input_url and save processed data to csv.

Args:
input_url(str|io):
Location of wind data.
output_path(str):
File location to save processed wind data.
"""