Now that you've gone through the Recurrent Neural Network lesson, I'll be teaching you what an LSTMis. This stands for Long Short Term Memory Networks, and are quite useful when our neural network needs to switch between remembering recent things, and things from long time ago. But first, I want to give you some great references to study this further. There are many posts out there about LSTMs, here are a few of my favorites:
- Chris Olah's LSTM post
- Edwin Chen's LSTM post
- Andrej Karpathy's lecture on RNNs and LSTMs from CS231n
Learn Gate
Forget Gate
Remember Gate
UseGate
If you would like to deepen your knowledge even more, go over the followingtutorial. Focus on the overview titled:Long Short-Term Memory Units (LSTMs).
If you are feeling confident enough, skip the overview and jump right into our next question:
The LSTM cell- taken form the Deep Learning tutorial
퀴즈 질문
The illustration above is of a LSTM cell. What would be the values of the three gates in situations where the cell retains information for a long period, without accepting a new input or producing an output?
The inputs gate: close to 0; the forget gate: close to 1; the output gate: close to 0
The inputs gate: close to 1; the forget gate: close to 1; the output gate: close to 0
The inputs gate: close to 0; the forget gate: close to 0; the output gate: close to 0
The inputs gate: close to 0; the forget gate: close to 1; the output gate: close to 1
퀴즈 답
1번