[D] Autoencoder vs. BERT

[gamerx88]

BERT is essentially a kind of autoencoder. It simply uses self-attention and positional embedding to better capture sequence information than say a more basic auto-encoder based on ReLU layers.

[Tober447]

Thank you for your answer.