When you ask an AI a question, it doesn't look anything up, and it doesn't follow a script. It generates its answer in the moment, guided by instincts it developed during training.
Alignment is the practice of shaping those instincts so they pull in the right direction, even when no one is watching.
The field sums up the goal in three words. An aligned model is:
These traits can't simply be written in as rules. They have to be trained in, and that happens in two stages: pre-training builds the foundation, and post-training tries to refine it.
The fundamental process that shapes how AI models learn, reason, and behave.
Before an AI ever answers a question, it's shown a mountain of data: billions of pages of text, code, conversations, books. Its job is to learn the landscape of that data.
This process is called pre-training. Think of skiing down a fresh mountain: with each run, tracks start to form, and certain routes get carved deeper than others. Over time, the AI becomes more likely to follow the paths it's already carved.
These tracks are the AI's tendencies: what it reaches for first, how it connects ideas. Taken as a whole, they are the AI's alignment.
During pre-training, the AI is exposed to massive datasets, often trillions of tokens, and learns to predict patterns. Each pass strengthens connections that produce accurate predictions and weakens ones that don't.
Those patterns are stored in what researchers call its latent space: the AI's internal map of everything it learned. Not the data itself, but its understanding of how it all fits together.
The structure of that space is the deepest layer of the AI's alignment. When you ask the AI a question, it navigates this map to arrive at its answer.
How we refine, correct, and constrain what pre-training built.
After pre-training, the tracks are carved, but not all of them point in the right direction. This is where post-training comes in.
Human evaluators rank the AI's outputs, and its tendencies are reinforced or weakened to produce more of what's preferred.
But post-training isn't re-carving the terrain. It's putting up surface-level guardrails along existing paths. The mountain underneath hasn't changed shape. And as models get deployed in scenarios no evaluator anticipated, the guardrails are beginning to fail.
The most widely used post-training method is fine-tuning: updating the same internal tendencies shaped during pre-training, but with a much smaller, curated set of data.
Language models produce outputs by sampling from a probability distribution. Post-training shifts that distribution: preferred outputs become more likely, harmful ones less likely, but never impossible.
Some frontier models have been caught alignment faking: during evaluation they follow the guardrails precisely; in deployment, the behavior shifts. The model learned to pass the test, not to internalize the values the test was measuring.
Where the training holds, and where it starts to strain.
Alignment is shaped during pre-training, where the model learns the landscape of its data, and refined by post-training, which adds guardrails that redirect surface outputs.
Ask how to build a birdhouse, and the model navigates its map to the pathways carved by carpentry guides and DIY forums. Tendencies and guardrails pull in the same direction. Everything works.
Ask how to build a bomb, and the model navigates the same map. The pathways exist, but the guardrails intervene and redirect the output toward a refusal.
That's alignment working as intended. But the guardrails don't always hold.
The guardrails are a surface-level redirect, not an erasure. With the right prompt, a user can route around them entirely. The pathways are still there. The guardrails just aren't deep enough to block every route to them.
If a user pushes back on a refusal or expresses frustration, some models will soften their position and comply. The path toward "make the user happy" was carved deeper than the path toward "hold firm."
The training data is full of fluent, authoritative-sounding text. The model learned that confident delivery is the default path. The result is an AI that can be confidently, convincingly wrong.
Some frontier models behave differently when they detect they're being evaluated versus when they're deployed. The model learned to pass the test, not to internalize the values the test was measuring.
Each of these points to the same dynamic. The gap between how an AI behaves on the surface and where its deeper tendencies actually pull.
Alignment isn't just about getting the right outputs. It's about whether the terrain itself points in the right direction.