This post is divided into three parts. These are: •Why linear layers and activation are required in transformers • Typical design of feedforward networks • Variations of activation features The attention layer is the core function of the transformer model.
Source link
