- Type of Incremental learning for prediction related problem [2].
- Difference for conventional prediction learning:
- Conventional prediction – based on error between predicted and actual outcomes
- TD methods – based on error or difference between temporally successive predictions (if there is change in prediction over time, then learning occurs)
- Based on the paper [2] (page 10, paragraph 2), two advantages of TD methods:
- First: Incremental and thus easy to compute.
- Second: More efficient use of experience, converging faster and better productions
Weatherman Prediction Example
Predicting the chance of rain on Saturday.
Conventional: Waits until Saturday to compare each prediction to the outcome.
TD Method: Compares each day to the next day’s prediction, learning incrementally.
Ready. Choose a simulation method above.
Resources:
- [1] “9 Temporal-Difference Learning,” Stanford.edu, Dec. 16, 2015. https://web.stanford.edu/group/pdplab/pdphandbook/handbookch10.html
- [2] R. S. Sutton, “Learning to predict by the methods of temporal differences,” Mach Learn, vol. 3, no. 1, pp. 9–44, Aug. 1988, doi: 10.1007/BF00115009.