Dictionary AI Technology
Attention mechanism
What it means
The part of a Transformer that decides which words matter most to each other - 'it' in a sentence gets linked back to the right noun.
An everyday example
At a noisy party, you can still lock onto the one voice saying your name. Attention lets the model 'lock onto' the words that matter.