These models have no explicit linguistic structure built into them, yet they may still implicitly learn structured relationships by attending to relevant tokens. Download a PDF of the paper titled Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages, by Paul Soulos and 11 other authors Download PDF Abstract:Machine translation has seen rapid progress with the advent of Transformer-based models.
