Concise Implementation in Recurrent Neural Networks

Hi @mli , I have a question about the hidden state of RNN with multiple hidden layers.

the hidden state is a list with the length of num_hidden_layers, however, Iā€™m wondering that why the shape of its element is (num_hidden_layers, batch_size, num_hidden_units), instead of (batch_size, num_hidden_units).


If there are multiple RNN layers, each layer will have a hidden state, here we just concat them together into a single ndarray.

1 Like