Deep Learning Articles
Catastrophic Forgetting
01 Dec 2018 •DEEP-LEARNING
TWEETSTORM
1/ Catastrophic Forgetting is a long-recognised problem in neural networks; and is of great interest in cognitive sciences. In plain words, it is the destructive interference effect of learning a new skill on pre-existing skills. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
2/ Research in Deep Learning has had a particular focus on this problem in recent time, particularly in the realm of reinforcement learning. e.g. [Rolnick, Ahuja, Schwarz et al. 2018], [Shin, Lee, Kim et al. 2017] among others. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018
.. Read More3/ A common thread in these research is the replay of past data to reinforce acquired skills from the past. Rolnick et al. (https://t.co/jluXqvN2Qr) choose a 50-50 split of replay vs. new task data. #deeplearning
— Sujith Jay Nair (@suj1th) December 1, 2018