What Are AI Model Weights? A Plain-English Guide

Model weights are the numbers a neural network learns during training, the values that decide how strongly each input signal gets multiplied and passed along as it flows through the network. A large language model with, say, 70 billion parameters has 70 billion of these numbers, and nothing else. There is no lookup table of facts, no database of sentences. Just weights, arranged in layers, doing multiplication and addition. When people ask

what a model "knows," they are really asking what its weights encode. This piece walks through what weights actually are, how one gets adjusted during training, and why the distinction between weights and parameters trips people up. It's one piece of the bigger picture of how AI models actually work.

Weights vs. AI model parameters: is there a difference?

Not really, in casual usage. "Parameters" is the umbrella term for every learnable number in a model, and weights make up the overwhelming majority of them. The rest are biases, a smaller set of numbers added after the multiplication step, one per neuron, that let the network shift its output up or down independent of the input. When a company says it shipped a "70B parameter model," that count includes both weights and biases, weights being the bulk of it.

A neuron, reduced to arithmetic

Strip away the diagrams and a single artificial neuron is a short equation. It takes some inputs, multiplies each one by a weight, adds a bias, and passes the result through a small nonlinear function. That's it. A whole model is millions or billions of these stacked in layers, each one feeding the next.

In a language model, those inputs start out as numeric representations of tokens, the chunks of text the model reads. For one neuron with two inputs x1 and x2, the math looks like this:

output = (x1 * w1) + (x2 * w2) + bias

w1 and w2 are the weights. Before training starts, they're just random numbers, usually small values seeded by chance. Training is the process of nudging them, over and over, until the neuron's output stops being random and starts being useful.

A worked example: one weight, one update

Here's the whole mechanism with real numbers, small enough to check by hand. Say our toy neuron has two inputs, x1 = 1.0 and x2 = 0.5, and it starts with these values:

• w1 = 0.20, w2 = 0.40, bias = 0.10

Run the numbers forward:

output = (1.0 * 0.20) + (0.5 * 0.40) + 0.10
       = 0.20 + 0.20 + 0.10
       = 0.50

Suppose the training example we're learning from says the correct output should be 0.80. Our neuron said 0.50, so it's off by 0.30. That gap is the error, and it's the entire reason training does anything at all. An algorithm called backpropagation works out how much each weight contributed to that error, and gradient descent uses that to decide which direction to move each weight to shrink it.

For this example, say the training step decides w1 needs to increase by 0.06 and w2 needs to increase by 0.03, a typical kind of adjustment when both inputs pushed the output in the same, too-low direction. The bias might tick up slightly too. After one update:

• w1 = 0.26 (was 0.20)

• w2 = 0.43 (was 0.40)

Run the same inputs through again:

output = (1.0 * 0.26) + (0.5 * 0.43) + 0.10
       = 0.26 + 0.215 + 0.10
       = 0.575

The output moved from 0.50 to 0.575, closer to the target of 0.80 but not there yet. That's the entire loop, run once. Real training repeats it across billions of examples and billions of weights at once, each nudge tiny. Nobody hand-picks the 0.06. An optimizer computes it fresh at every step, based on how much that weight is to blame for the current error. Multiply this one-neuron story by however many neurons and layers a real model has, and that is the entire training process. Nothing more mysterious is happening underneath a language model's fluent output. The scale is what's hard to picture, not the mechanism.

So this answers what does training an ai model do

Training does exactly one thing, over and over: it compares the model's current output to what the output should have been, then adjusts every weight a small amount in the direction that would have reduced that specific error. Do that across a large enough dataset, enough times, and the weights settle into values that generalize, meaning they produce reasonable outputs even on inputs the model never saw. That settling is what people mean when a model has "learned" something. It hasn't memorized a rulebook. It has found a set of numbers that, multiplied through in the right order, tend to produce useful outputs.

What weights are not

• Not a database. There's no row you can query for "capital of France." The knowledge is smeared across millions of weights that happen to produce the right token when the right pattern of prior tokens shows up.

• Not fixed for the life of a product. A new checkpoint, meaning a new saved snapshot of all the weights, is what a "model update" actually ships, and it's exactly why a model can get worse after an update.

• Not interpretable one at a time. Looking at a single weight's value tells you almost nothing. Meaning lives in the pattern across many weights together.

Neural network weights explained, in one sentence

Neural network weights explained as briefly as possible: they are the learned multipliers that turn an untrained, randomly-initialized network into one that produces useful outputs, and everything a model can do lives in the specific values those multipliers ended up at.

Why this matters if you're not training models yourself

Most people building on top of AI never touch a weight directly, but the concept explains things you do run into: why a large model's file is tens of gigabytes (mostly a giant list of weight values), and why fine-tuning is possible without retraining from scratch, since you're adjusting existing weights rather than starting from random ones, as in approaches like RLHF. Push those adjustments too aggressively and you get catastrophic forgetting, where new training overwrites weights the model needed for something it used to do well.

FAQ

Are weights the same as parameters?

Nearly. Parameters is the broader term covering both weights and biases, but weights make up the vast majority of a model's parameter count, so the terms get used interchangeably in casual conversation.

How many weights does a typical AI model have?

It ranges enormously, from a few million in a small classifier to hundreds of billions in a large language model. The count is usually what people mean when they cite a model's parameter size, like "8B" or "70B."

Can you view or edit a model's weights directly?

Technically yes, weights are stored as files you can open, but a single weight in isolation is meaningless. Editing one by hand won't produce a predictable change, since meaning is distributed across many weights acting together.

Do weights change every time a model is used?

No. Weights are fixed once training finishes, which is what makes a model a specific, deployable checkpoint. Using the model, meaning inference, only reads the weights. Only further training changes them.

What is the difference between weights and activations?

Weights are learned and stored permanently in the model file. Activations are the temporary numbers produced at each layer while processing a given input, and they disappear once that pass finishes. Weights persist, activations don't.