Autoregressive Models

Interactively explore how autoregressive models generate sequences one token at a time, using context to predict the next element.

Autoregressive Sequence Simulation

Educational Autoregressive Sandbox

Explanation

  • Concept: Generates data sequentially, one element at a time, based on previous elements.
  • Example: GPT models predict the next word in a sequence using all previous words.
  • PixelRNN/PixelCNN: Early autoregressive models for images, predicting pixels based on neighbors.
Click Generate Next Token to extend the sequence. Probability bars show the model's prediction for the next token.
Next ▶