While attention mechanism helps to attend to all the information at the same time, it loses the order of the text. The order of the text matters like "The dog chased the cat" and "the cat chased the dog" - is not same thing.

To add some sense of the order, in the transformer paper, the authors have added the positional encoding to the word embedding.

There are 2 kinds of positional encoding:

1. Fixed Encoding: In the transformer paper, the authors have used the sine and cosine information for the positional encoding. Learn more from Why Trigonometric Function for Positional Encoding?

2. Learned Embedding: In the learned embedding, like the word embedding, the models learn about the position. So there is another extra embedding layer for the position

With the positional encoding, added to the word embedding, the model can simultaneously look at different tokens with their order embedded into the representation.

References