Bidirectional RNN or LSTM
Unlike standard RNN or LSTM, bidirectional ones read input from both sides, left to right and right to left. They have two separate hidden layers for each of them and both of the hidden layer output goes to the output layer.