Deep Learning Articles

Catastrophic Forgetting

01 Dec 2018 • DEEP-LEARNING  TWEETSTORM 

1/ Catastrophic Forgetting is a long-recognised problem in neural networks; and is of great interest in cognitive sciences. In plain words, it is the destructive interference effect of learning a new skill on pre-existing skills. #deeplearning

— Sujith Jay Nair (@suj1th) December 1, 2018

2/ Research in Deep Learning has had a particular focus on this problem in recent time, particularly in the realm of reinforcement learning. e.g. [Rolnick, Ahuja, Schwarz et al. 2018], [Shin, Lee, Kim et al. 2017] among others. #deeplearning

— Sujith Jay Nair (@suj1th) December 1, 2018

3/ A common thread in these research is the replay of past data to reinforce acquired skills from the past. Rolnick et al. (https://t.co/jluXqvN2Qr) choose a 50-50 split of replay vs. new task data. #deeplearning

— Sujith Jay Nair (@suj1th) December 1, 2018

.. Read More