How LLMs actually work

you type “what’s the capital of France?” into a chatbot and it answers “Paris”. it feels like it knows 🧠

most explanations of what happens in between either stop at “it predicts the next word” or jump straight into papers full of notation. this post sits in the middle. it goes from raw text all the way to the answer, with the maths, and i tried to keep it readable even if you’ve never touched machine learning.

it’s a condensed version of the background i wrote for my MSc thesis on LLM security. let’s dig in.

one probability

the whole thing boils down to one question. given the text so far, how likely is each possible next token?

$$P(\text{Paris} \mid \text{The capital of France is}) \approx 0.32$$

read it as “the probability of Paris given The capital of France is”. the $\mid$ means “given”. a model doesn’t output just “Paris”, it outputs a probability for every token it knows, and Paris gets the biggest share because it’s the right continuation.

it only gets about a third, though, because a sentence can go on in lots of ways. the runners-up are blanks like ______ (11%), probably from all the fill-in-the-blank quizzes the model has read. a smaller, older model like GPT-2 only gives Paris 3%, and we’ll see why size matters at the end.

to write more than one word, you apply this over and over. the probability of a whole sequence factorises into a product of next-token probabilities, which is just the chain rule of probability.

$$P(x_1, \dots, x_n) = \prod_{t=1}^{n} P(x_t \mid x_1, \dots, x_{t-1})$$

models that generate text this way, one token at a time, each conditioned on everything before it, are called autoregressive.

so the rest of this post answers one question. how do you compute $P(x_t \mid x_1, \dots, x_{t-1})$?

tokens

first problem, a model is a big pile of maths and has no idea what a word is. text has to become numbers. it gets cut into tokens, and each token has an ID in a fixed vocabulary.

a token is roughly a word. but “roughly” is doing a lot of work, and the details surprised me when i ran real tokenizers on a few examples with OpenAI’s tiktoken.

text	GPT-2	GPT-4	GPT-4o
unbelievable	un bel iev able	un belie vable	un bel ievable
␣unbelievable	␣unbelievable	␣unbelievable	␣unbelievable
bonjour	bon j our	bon jour	bonjour
␣Paris	␣Paris (ID 6342)	␣Paris (ID 12366)	␣Paris (ID 12650)

␣ is a space. a few things to notice:

• unbelievable is not cut into un + believable like you’d guess. the splits don’t follow grammar at all.
• with a space in front, it’s a single token in all three. the space is part of the token, and ␣unbelievable (in the middle of a sentence) is probably much more common in training text than unbelievable at the start of one.
• newer tokenizers have bigger vocabularies (about 200K entries for GPT-4o, 50K for GPT-2), so more whole words get their own token. bonjour is three tokens for GPT-2 and one for GPT-4o.
• the same word has a different ID in each model. an ID only means something for the tokenizer that produced it.

what to remember is that a token is about a word, give or take.

where does the vocabulary come from?

i always assumed someone wrote the vocabulary by hand. nope 😅 GPT tokenizers use byte pair encoding (BPE), brought to NLP by Sennrich et al. in Neural Machine Translation of Rare Words with Subword Units. the algorithm is almost disappointingly simple:

1. split every word into single characters
2. count every pair of adjacent symbols in the corpus
3. merge the most frequent pair into a new symbol
4. repeat until the vocabulary is big enough

here it is on a tiny corpus.

after 6 merges, newest and widest share est, and low is a single token. real tokenizers do exactly this on huge amounts of text until they reach their vocabulary size, 50,257 tokens for GPT-2 and around 200K for GPT-4o.

embeddings

we now have a list of integers, [976, 9029, 328, 10128, 382] for “The capital of France is” with GPT-4o. that’s still not something you can do maths on, since token 10128 isn’t “bigger” than token 976 in any useful way.

so each ID is mapped to a vector, its embedding. the model has a big matrix $E$ with one row per token in the vocabulary, and looking up a token just means taking its row.

$$E \in \mathbb{R}^{|V| \times d}, \qquad e_{\text{hello}} = E_{\text{id(hello)}} = [0.2,\ -0.4,\ \dots]$$

$|V|$ is the vocabulary size and $d$ the dimension of each vector. and $d$ is big. here are the numbers for Llama 3, from Meta’s paper The Llama 3 Herd of Models.

model	$d$	blocks
Llama 3 8B	4,096	32
Llama 3 70B	8,192	80
Llama 3 405B	16,384	126

Llama 3 has a vocabulary of about 128K tokens, so for the 8B model that’s a lookup table of roughly 128K × 4,096, about half a billion numbers just to turn IDs into vectors.

nobody picks these numbers by hand either. they’re learned during training (we’ll get there), and the interesting part is that the geometry of the vectors ends up meaning something.

measuring closeness with cosine similarity

tokens that are used in similar ways end up with vectors pointing in similar directions. the standard way to measure that is cosine similarity, the cosine of the angle $\theta$ between two vectors:

$$\cos(\theta) = \frac{a \cdot b}{\lVert a \rVert \, \lVert b \rVert} = \frac{\sum_i a_i b_i}{\sqrt{\sum_i a_i^2} \, \sqrt{\sum_i b_i^2}}$$

1 means same direction, 0 means perpendicular (unrelated), −1 means opposite. here’s the calculation by hand.

now try a few pairs.

two things i found interesting here.

opposites are not opposite vectors. you’d expect hot and cold to point in opposite directions, with a cosine close to −1. they don’t. their cosine is 0.45, which is actually more similar than hello and cat (0.31). embeddings are learned from context, and hot and cold show up in exactly the same kind of sentences (“it’s really ___ today”). linguists had that idea long before neural networks, “you shall know a word by the company it keeps”, as J. R. Firth put it in 1957. also, unrelated words don’t score 0 in practice. with this model they sit around 0.3.

directions carry meaning. in the famous word2vec example, $e_{\text{king}} - e_{\text{man}} + e_{\text{woman}}$ lands closest to $e_{\text{queen}}$. LLMs learn their embeddings differently, but the idea that meaning lives in the geometry is the same.

what about word order?

one more thing. as we’ll see, the model processes all tokens in parallel, so nothing tells it that cat comes before it. position has to be added explicitly. the original transformer adds a sinusoidal “position vector” to each embedding. Llama instead rotates the vectors used inside attention by an angle that depends on the position, a trick called RoPE.

neural networks, quickly

before the transformer itself, a quick detour. an LLM is a neural network, a very big one. so what’s a neural network?

the building block is a neuron. it takes some inputs, multiplies each one by a weight, adds a bias, and passes the result through a non-linear function $f$.

$$y = f\left(\sum_{i=1}^{n} w_i x_i + b\right)$$

with numbers, take inputs $x = [1.0,\ 0.5,\ -1.0]$, weights $w = [0.8,\ -0.2,\ 0.4]$ and bias $b = 0.1$.

$$z = 0.8 \times 1.0 + (-0.2) \times 0.5 + 0.4 \times (-1.0) + 0.1 = 0.4$$

and with $f(z) = \text{ReLU}(z) = \max(0, z)$, we get $y = 0.4$. that’s it, one neuron is trivial.

the trick is scale. put many neurons side by side and you get a layer, which is nothing more than a matrix multiplication.

$$y = f(Wx + b)$$

stack layers and you get a network. all the $W$ and $b$ are the parameters, the numbers training will adjust. and since it’s all matrix multiplications, it’s exactly the kind of work GPUs are built for.

the non-linear $f$ matters. without it, stacking layers would collapse into one big matrix multiplication, and the network could only model linear relationships.

the transformer

now the actual architecture, from the 2017 paper Attention Is All You Need. GPT-style LLMs only keep the “decoder” half of it, a stack of identical blocks applied one after the other.

every block has two main parts. attention is where tokens exchange information, and the MLP is where each token is processed on its own. let’s take them one at a time.

attention

take this sentence.

The cat is big, it must have eaten a lot.

when the model gets to it, the vector for it alone is useless, since “it” could refer to anything. to predict what comes next, the model needs to figure out that it is the cat. attention is the mechanism that lets a token pull information from other tokens. click on a few tokens.

this is one of GPT-2’s 144 attention heads, the one where it looks the most at cat. notice two things. the weights for a token always add up to 100%, and a token can only look at itself and the tokens before it. that’s the “masked” in masked attention, because when you’re learning to predict the next token, you can’t be allowed to peek at it.

not every head is that clean, though. averaged over the 12 heads of that block, it sends almost half of its attention (48%) to the very first token, The. models tend to use the first token as a parking spot when nothing else is worth looking at, a quirk known as an attention sink.

so how are these weights computed? here’s the formula from the paper.

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$

this one wasn’t clear to me at all at first, so let’s unpack it for a single token.

one of these computations is called a head. models run many heads in parallel, each with its own $W_Q$, $W_K$ and $W_V$, so different heads can track different kinds of relationships. Llama 3 8B has 32 heads in every block.

the rest of the block

after attention, each token goes through an MLP (multilayer perceptron). it’s the neural network from earlier, two big layers applied to every token independently.

$$\text{MLP}(x) = W_2\, f(W_1 x)$$

the hidden layer is wider than $d$, 4 times wider in the original transformer and 14,336 for the 4,096-dimensional vectors of Llama 3 8B. a common way to think about it is that attention moves information between tokens, and the MLP processes it.

two more ingredients hold everything together.

residual connections. each part adds its result to its input instead of replacing it (the coloured arrows in the diagram). it’s what makes stacks of dozens of blocks trainable.

$$x \leftarrow x + \text{Attention}(\text{Norm}(x))$$ $$x \leftarrow x + \text{MLP}(\text{Norm}(x))$$

normalisation. it rescales each vector so values don’t drift or explode across dozens of blocks. the original transformer used LayerNorm after each part; most recent models, Llama included, put a norm at the start of each part instead, which is the layout in the diagram.

parallel inside, sequential across

inside a block, every token goes through attention and the MLP at the same time. it’s matrix multiplications over the whole sequence, and the mask makes sure no token cheats by looking ahead. this parallelism is a big reason why transformers replaced recurrent networks, which had to read tokens one by one.

from one block to the next, though, it’s sequential. block 2 works on the output of block 1, and so on. each block refines the representation a little more, and by the end, the vector at the last position has soaked up everything the model got from the prompt.

the output

after the last block (and a final norm), we only need the vector at the last position, call it $h$. a final linear layer turns it into one score per token in the vocabulary, the logits.

$$z = W_{\text{out}}\, h \qquad z \in \mathbb{R}^{|V|}$$

and a softmax turns those scores into probabilities. that’s our $P(x \mid \text{prompt})$ from the very beginning 🎉

$$P(x_i \mid \text{prompt}) = \frac{e^{z_i / T}}{\sum_{j} e^{z_j / T}}$$

$T$ is the temperature. with $T = 1$ it’s a regular softmax. play with it.

the first 5 bars are the model’s favourite tokens (␣ is a space, ______ a blank like in a quiz, ↵ a line break), and the last one is the other 151,931 tokens of its vocabulary put together. at a low temperature, Paris takes everything. at 2, the distribution is so flat that the 151,931 others share 94% between them. that’s how you get more creative, or completely off-topic, text.

picking a token

the model gives probabilities, not a word, so something has to choose. always taking the most likely token (greedy decoding) sounds like the right thing to do, but it tends to produce bland, repetitive text. so chatbots usually sample from the distribution, often after cutting off the unlikely tail. that’s why the same prompt can give different answers.

then the chosen token is added to the input, and the whole model runs again.

training

everything so far is inference, using a model that already has good parameters. but where do $E$, $W_Q$, $W_K$, $W_V$, the MLP matrices and all the others come from? they start as random numbers, and training adjusts them.

how many are there? that’s the “8B” in Llama 3 8B, 8 billion parameters. the biggest open models are in the hundreds of billions, 405B for the largest Llama 3.

predicting the next token, again

the nice part is that no labelled data is needed. take any text, hide the next token, and ask the model to predict it. the text is its own answer key. the model reads “The capital of France is”, outputs probabilities, and we look at the probability it gave to the token that actually came next. that gives the loss for that token.

$$\mathcal{L} = -\log P(x_t \mid x_1, \dots, x_{t-1})$$

if the model gave the right token a probability of 0.9, the loss is $-\ln 0.9 \approx 0.105$, small. if it gave 0.01, the loss is $-\ln 0.01 \approx 4.6$, big. and as the probability goes to 0, the loss goes to infinity, so being confidently wrong is punished hard. this is the cross-entropy loss, averaged over every token of every text in a batch.

gradient descent

to reduce the loss, we compute its gradient $\nabla_\theta \mathcal{L}$, which tells us, for every parameter, which direction makes the loss go up. then we take a small step the other way.

$$\theta \leftarrow \theta - \eta \, \nabla_\theta \mathcal{L}$$

$\theta$ is all the parameters and $\eta$ is the learning rate, the size of the step. computing this gradient efficiently for billions of parameters is what backpropagation does, by applying the chain rule layer by layer, from the loss back to the input. real training uses smarter optimizers like Adam, but the idea is the same.

then repeat. a lot. Llama 3 was pre-trained on more than 15 trillion tokens, and the 405B model took about $3.8 \times 10^{25}$ floating point operations.

bigger is better, with conditions

more parameters generally means a better model, but parameters alone aren’t enough. DeepMind’s Chinchilla paper found that for a fixed compute budget, model size and training data should grow together, and their 70B model trained on 1.4T tokens beat the 280B Gopher.

from text predictor to assistant

a pre-trained model only continues text. ask it a question and it might continue with three more questions. chat assistants come from post-training, fine-tuning on example conversations, then reinforcement learning from human feedback (RLHF). underneath, though, it’s still the same next-token machine.

why this matters for security

this is the part i care about the most, and what my MSc thesis focused on. every piece above has a security side.

• tokens. the model reads tokens, not what you see on screen. invisible Unicode characters still become tokens, which is enough to hide instructions inside text that looks harmless, a trick known as ASCII smuggling.
• attention. the system prompt written by the developer, the user’s message and the web page the model was asked to summarise all end up in one flat sequence, going through the same attention. nothing in the maths says “these tokens are instructions, those are just data”. that’s why prompt injection is so hard to fix, and Greshake et al. showed it on real applications in Not what you’ve signed up for.
• sampling. the same prompt can give different answers, so a guardrail that stopped an attack once can let it through on the next try.
• training. everything the model knows comes from its training data and its post-training, so whoever can slip documents into that data can shape what it does.

conclusion

from the outside LLMs look like magic, but once you take them apart it’s a stack of fairly simple ideas: tokens, vectors, dot products, softmax, and a lot of gradient descent at a scale that’s hard to picture. i hope this gives you a clearer idea of what’s going on, and makes the papers easier to read. if you want to go further, The Illustrated Transformer by Jay Alammar has lots more diagrams, nanoGPT by Andrej Karpathy is a GPT small enough to read in an afternoon, and after this post the original paper should be a lot less scary.

what’s next

these fundamentals are what my next articles will build on, as they go deeper into LLMs and cyber security.

if you spot a mistake, or know a better way to explain one of these parts, i’d love to hear about it!