Local Attention
Local attention is one kind of Attention where each word doesn't attend to all other words simultaneously but a specific region of input sentence.
One example is Sliding Window Attention
Local attention is one kind of Attention where each word doesn't attend to all other words simultaneously but a specific region of input sentence.
One example is Sliding Window Attention