I am working on a small project in the lab with an Arduino Mega 2560 board. I want to average the signal (voltage) of the positive-slope portion (rise) of a triangle wave to try to remove as much noise as possible. My frequency is 20Hz and I am working with a data rate of 115200 bits/second (fastest recommended by Arduino for data transfer to a computer).

The raw signal looks like this:

Raw Signal

Raw Signal Zoom

Raw Signal Zoom Plus

My data is stored in a text file, with each line corresponding to a data point. Since I do have thousands of data points, I expect that some averaging would smooth the way my signal looks and make a close-to-perfect straight line in this case. However, other experimental conditions might lead to a signal where I could have features along the positive-slope portion of the triangle wave, such as a negative peak, and I absolutely do need to be able to see this feature on my averaged signal.

I am a Python beginner so I might not have the ideal approach to do so and my code might look bad for most of you. I would still like to get your hints / ideas on how I can improve my signal processing code to achieve a better noise removal by averaging the signal.

import matplotlib.pyplot as plt
import math

data_filename = "My_File_Name"
filepath = "My_File_Path" + data_filename + ".txt"

# Open the Raw Data
with open(filepath, "r") as f:
    rawdata = f.readlines()

# Remove the \n     
rawdata = map(lambda s: s.strip(), rawdata)

# Plot the Raw Data
plt.plot(rawdata, 'r-')
plt.ylabel('Lightpower (V)')

# Number of data points for each range 
datarange = 15 # This number can be changed for better processing
max_i_range = int(math.floor(len(rawdata)/datarange))-3

#Declare an empty lists for the max and min
min_list = []
max_list = []

min_list_index = []
max_list_index = []


for i in range(0, max_i_range):
    delimiter0 = i * datarange
    delimiter1 = (i+1) * datarange
    delimiter2 = (i+2) * datarange
    delimiter3 = (i+3) * datarange

    sumrange1 = sum(float(rawdata[i]) for i in range(delimiter0, delimiter1 + 1))
    averagerange1 = sumrange1 / len(rawdata[delimiter0:delimiter1])
    sumrange2 = sum(float(rawdata[i]) for i in range(delimiter1, delimiter2 + 1))
    averagerange2 = sumrange2 / len(rawdata[delimiter1:delimiter2])
    sumrange3 = sum(float(rawdata[i]) for i in range(delimiter2, delimiter3 + 1))
    averagerange3 = sumrange3 / len(rawdata[delimiter2:delimiter3])

    # Find if there is a minimum in range 2
    if ((averagerange1 > averagerange2) and (averagerange2 < averagerange3)):
        min_list.append(min(rawdata[delimiter1:delimiter2])) # Find the value of all the minimum 

        #Find the index of the minimum
        min_index = delimiter1 + [k for k, j in enumerate(rawdata[delimiter1:delimiter2]) if j == min(rawdata[delimiter1:delimiter2])][0] # [0] To use the first index out of the possible values

    # Find if there is a maximum in range 2
    if ((averagerange1 < averagerange2) and (averagerange2 > averagerange3)):
        max_list.append(max(rawdata[delimiter1:delimiter2])) # Find the value of all the maximum

        #Find the index of the maximum
        max_index = delimiter1 + [k for k, j in enumerate(rawdata[delimiter1:delimiter2]) if j == max(rawdata[delimiter1:delimiter2])][0] # [0] To use the first index out of the possible values


# One rise pattern goes from a min to a max
numb_of_rise_pattern = 50 # This number can be increased or lowered. This will average 50 rise patterns

max_min_diff_total = 0

for i in range(0, numb_of_rise_pattern):
    max_min_diff_total = max_min_diff_total + (max_list_index[i]-min_list_index[i])

# Find the average number of points for each rise pattern
max_min_diff_avg = abs(max_min_diff_total / numb_of_rise_pattern)

# Find the average values for each of the rise pattern
avg_position_value_list = []

for i in range(0, max_min_diff_avg):

    sum_position_value = 0
    for j in range(0, numb_of_rise_pattern):

        sum_position_value = sum_position_value + float( rawdata[ min_list_index[j] + i ] )
    avg_position_value = sum_position_value / numb_of_rise_pattern


#Plot the Processed Signal
plt.plot(avg_position_value_list, 'r-')
plt.ylabel('Lightpower (V)')

At the end, the processed signal looks like this:

Averaged Signal

I would expect a straighter line, but I could be wrong. I believe that there are probably a lot of flaws in my code and there would certainly be better ways to achieve what I want.

I have included a link to a text file with some raw data if any of you want to have fun with it.

  • \$\begingroup\$ Welcome to Code Review! Great job on your first question! \$\endgroup\$
    – SirPython
    Feb 23, 2016 at 22:03
  • 1
    \$\begingroup\$ Wait, why should the processed signal look like a straight line with a positive slope? I'm not sure what this algorithm is trying to do I guess. \$\endgroup\$ Feb 23, 2016 at 22:18
  • \$\begingroup\$ That file-sharing site's popups are obnoxious. \$\endgroup\$ Feb 23, 2016 at 22:22
  • \$\begingroup\$ The signal sent is a triangle wave. I am trying to extract only one of the two portion of the triangle wave (the one with the positive slope) and average it to remove as much noise as possible. In the above code, I take 50 different triangle profiles and average the portion with the positive slope to obtain what should look like a line with a positive slope with the least noise as possible. Sorry if it's not clear, I didn't mean a line with 0 slope (straight line). \$\endgroup\$
    – LaGuille
    Feb 23, 2016 at 22:23
  • \$\begingroup\$ Can you add some more context about the use-case for this? It isn't clear to me whether there's some contextual reason for you to average multiple waveforms rather than low-pass filtering your samples. \$\endgroup\$ Feb 23, 2016 at 22:42

2 Answers 2


As David Morris indicates, it might be simpler to use a filtering/smoothing function, such as a moving window average. This is pretty simple to implement using the rolling function from pandas.Series. (Only 501 points are shown.) Tweak the numerical argument (window size) to get different amounts of smoothing.

import pandas as pd
import matplotlib.pyplot as plt

# Plot the Raw Data
ts = rawdata[0:500]
plt.plot(ts, 'r-')
plt.ylabel('Lightpower (V)')

# original version
# smooth_data = pd.rolling_mean(ts,5).plot(style='k')
# pandas has changed; so, here's the new version:
smooth_data = pd.Series(ts).rolling(window=7).mean().plot(style='k')

enter image description here (Moving average)

A moving average is, basically, a low-pass filter. So, we could also implement a low-pass filter with functions from SciPy as follows:

import scipy.signal as signal

# First, design the Butterworth filter
N  = 3    # Filter order
Wn = 0.1 # Cutoff frequency
B, A = signal.butter(N, Wn, output='ba')
smooth_data = signal.filtfilt(B,A, rawdata[0:500])

enter image description here (Low-Pass Filter)

This filter method is from OceanPython.org BTW.

  • \$\begingroup\$ thank you for this answer, I was going to try the code, but what is the ts in the first code ? \$\endgroup\$ Feb 13, 2020 at 9:05
  • \$\begingroup\$ Thanks for the note. I intended ts to be a subset of the raw data. I've modified my answer to make this clear. BTW, as the comments show, the pandas module has changed from rolling_mean to rolling as an attribute of pandas.Series. I've fixed this. \$\endgroup\$ Feb 18, 2020 at 1:27
  • Convert data to float once and forever:

    with open(filepath, "r") as f:
        rawdata = [float(line.strip()) for line in f.readlines()]
  • len(rawdata[delimiter0:delimiter1]) equals to delimiter1 - delimiter0. I suppose the latter is somewhat cleaner.

  • Detecting slopes is questionable. It seems that it introduces a phase shift. I recommend to add a second step: since you know it must be a straight line, min-square interpolate what you found at the first step.

    In any case it should be factored out into a function.

  • Averaging slopes is very questionable. Each slope may have different number of samples, and you account for that by computing an average number. This means that shorter slopes will contribute data which actually belong to the falling slope, and longer slopes do not contribute their trailing parts. You may want to resample them.

    I also suspect that the slopes not being aligned correctly (see above) contributes a lot.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.