“By tracking this contextual information, transformers can handle longer strings of text and capture the meanings of words more accurately. For example, ‘hot dog’ means very different things in the sentences ‘Hot dogs should be given plenty of water’ and ‘Hot dogs should be eaten with mustard.“
– Will Douglas Heaven
Large language models (LLMs) are made with large computing power. ChatGPT‘s roots extend back to research in the ’80s and ’90s.
MIT Technology Review traces the evolution of LLMs and points to the invention of transformers as the key breakthrough, in 2017.
Transformers provide context for text generation. They are the foundation for ChatGPT and the many versions of LLMs emerging today.
ChatGPT is everywhere. Here’s where it came from
MIT TECHNOLOGY REVIEW | February 8, 2023 | by Will Douglas Heaven