Bidirectional LSTM model state output data format

The document shows the bidirectional LSTM model output hidden state and cell of memory in the formation of [2*num_layers, batch_size, num_hidden]. I am wondering what the sequence in the first dimension.

Suppose the num_layers = 2, is the sequence of hidden state in the following way:

  1. the forward hidden state of the first layer,
  2. the forward hidden state of the second layer,
  3. the backward hidden state of the first layer
  4. the backward hidden state of the second layer?

(kwargs[’{}{}{}{}’.format(d, l, g, t)].reshape(-1)
for t in [‘weight’, ‘bias’]
for l in range(self._num_layers)
for d in [‘l’, ‘r’][:self._dir]
for g in [‘i2h’, ‘h2h’])

If you look at the code in rnn_layer.py, it uses this generator to order the parameters, and then concats it together to represent the state. So yes. Your assumption is correct.

1 Like

Thank you a lot for your answer @keerthanvasist !