Catastrophic Forgetting
01 Dec 2018 •DEEP-LEARNING TWEETSTORM 1/ Catastrophic Forgetting is a long-recognised problem in neural networks; and is of great interest in cognitive sciences. In plain words, it is the destructive interference effect of learning a new skill on pre-existing skills. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
2/ Research in Deep Learning has had a particular focus on this problem in recent time, particularly in the realm of reinforcement learning. e.g. [Rolnick, Ahuja, Schwarz et al. 2018], [Shin, Lee, Kim et al. 2017] among others. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
3/ A common thread in these research is the replay of past data to reinforce acquired skills from the past. Rolnick et al. (https://t.co/jluXqvN2Qr) choose a 50-50 split of replay vs. new task data. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
4/ The use of replay using a limited buffer demonstrates a drastic reduction in catastrophic forgetting. The limited replay buffer result, in particular, is an exciting one; considering that deep learning models are such resource-hogs. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
5/ This research, in cumulation with its predecessors, answers an important question about replay: a limited sample of past experiences produces a minimal difference in performance in comparison to an unlimited buffer. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
6/ An important direction this experiment could take is via tweaks on the split of replay vs new tasks data, and uniform reservoir sampling on the replay buffer. Anybody who has tried to learn using spaced repetition knows that a uniform split at every point in the...
— Sujith Jay Nair (@suj1th) December 1, 2018
7/ ...future does not guarantee optimality (or, so they say). A least-recently-used bias in sampling should help in achieving a spaced-repetition kind of behavior in replay. It would be interesting to see the effects of it on catastrophic forgetting. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018