@abucci@anthony.buc.ci Can you explain the type of neural networks behind these *GPT(s) and how they differ from more traditional ANNs? 🙏

@abucci@anthony.buc.ci Interestingly the Wikipedia article on GPT-3 describe it as:

Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model released in 2020 that uses deep learning to produce human-like text. Given an initial text as prompt, it will produce text that continues the prompt.

Which is even more confusing to me, mostly because it doesn’t speak of a neural network at all. Basically I was (on my short-lived holiday) doing some R&D on neural networks, evolutionary algorithms and other reading 😅

I tried to read up on autoregressive language models(s) btw, and gave up. Way over my puny head 🤦‍♂️

@abucci@anthony.buc.ci Noice! 👌 Bwtween you and my reading I have a muumuu deeper understanding of this shit 🙇‍♂️

Sasly I didn’t come across RNNS though 😆 But yhay doesn’t matter 🤔