[python] Pandas: rolling mean by time interval

This example seems to call for a weighted mean as suggested in @andyhayden's comment. For example, there are two polls on 10/25 and one each on 10/26 and 10/27. If you just resample and then take the mean, this effectively gives twice as much weighting to the polls on 10/26 and 10/27 compared to the ones on 10/25.

To give equal weight to each poll rather than equal weight to each day, you could do something like the following.

>>> wt = df.resample('D',limit=5).count()

            favorable  unfavorable  other
enddate                                  
2012-10-25          2            2      2
2012-10-26          1            1      1
2012-10-27          1            1      1

>>> df2 = df.resample('D').mean()

            favorable  unfavorable  other
enddate                                  
2012-10-25      0.495        0.485  0.025
2012-10-26      0.560        0.400  0.040
2012-10-27      0.510        0.470  0.020

That gives you the raw ingredients for doing a poll-based mean instead of a day-based mean. As before, the polls are averaged on 10/25, but the weight for 10/25 is also stored and is double the weight on 10/26 or 10/27 to reflect that two polls were taken on 10/25.

>>> df3 = df2 * wt
>>> df3 = df3.rolling(3,min_periods=1).sum()
>>> wt3 = wt.rolling(3,min_periods=1).sum()

>>> df3 = df3 / wt3  

            favorable  unfavorable     other
enddate                                     
2012-10-25   0.495000     0.485000  0.025000
2012-10-26   0.516667     0.456667  0.030000
2012-10-27   0.515000     0.460000  0.027500
2012-10-28   0.496667     0.465000  0.041667
2012-10-29   0.484000     0.478000  0.042000
2012-10-30   0.488000     0.474000  0.042000
2012-10-31   0.530000     0.450000  0.020000
2012-11-01   0.500000     0.465000  0.035000
2012-11-02   0.490000     0.470000  0.040000
2012-11-03   0.490000     0.465000  0.045000
2012-11-04   0.500000     0.448333  0.035000
2012-11-05   0.501429     0.450000  0.032857
2012-11-06   0.503333     0.450000  0.028333
2012-11-07   0.510000     0.435000  0.010000

Note that the rolling mean for 10/27 is now 0.51500 (poll-weighted) rather than 52.1667 (day-weighted).

Also note that there have been changes to the APIs for resample and rolling as of version 0.18.0.

rolling (what's new in pandas 0.18.0)

resample (what's new in pandas 0.18.0)

Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to pandas

xlrd.biffh.XLRDError: Excel xlsx file; not supported Pandas Merging 101 How to increase image size of pandas.DataFrame.plot in jupyter notebook? Trying to merge 2 dataframes but get ValueError Python Pandas User Warning: Sorting because non-concatenation axis is not aligned How to show all of columns name on pandas dataframe? Pandas/Python: Set value of one column based on value in another column Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Python convert object to float

Examples related to time-series

How to convert dataframe into time series? Peak signal detection in realtime timeseries data Pandas: rolling mean by time interval Excel plot time series frequency with continuous xaxis How to calculate rolling / moving average using NumPy / SciPy? Plotting two variables as lines using ggplot2 on the same graph