training text
corpusgradient descent on next-token prediction
loss—
perplexity—
|grad|—
tokens0
sample generated while training
tokenization
byte-pair encodingtext → integers
The model never sees letters. A BPE tokenizer is learned from the corpus:
start from single characters, then repeatedly merge the most frequent adjacent pair.
1 · vocabulary
2 · learned merges
3 · prompt, encoded
4 · token frequency in corpus
transformer
—every intermediate tensor of one live forward pass
value scale
−max 0 +max
weights & biases
—one pixel per parameter · hover to identify
tensors
selected tensor
architecture
input
prompttype to run a forward pass
Every keystroke runs a fresh forward pass, so the transformer panel
always shows what the model is doing with the text above.
temperature reshapes the distribution before sampling — 0 always takes the most likely token, higher values flatten the odds. top-k throws away everything outside the k best candidates.
temperature reshapes the distribution before sampling — 0 always takes the most likely token, higher values flatten the odds. top-k throws away everything outside the k best candidates.
output
0 tokenssampled one token at a time
next-token distribution