Training vs Inference
Training is the process of teaching a model by feeding it large amounts of data, inference is the process of actually using that already-trained model to produce an answer. Training happens once or periodically. Inference happens every time someone uses the model.
Model Training
Model training is how an AI model learns: it sees many examples, checks its guesses against the right answers and adjusts itself until the errors shrink.
Inference
Inference is the moment a trained AI model does its job: it takes a new question and produces an answer. It happens on every request, so it is the ongoing cost.
Before the model is deployed, often once or periodically
Every single time a user asks a question or makes a request
Large one-time or periodic cost
Smaller cost repeated per use, adds up at scale
Build the model's knowledge and patterns
Apply that knowledge to a real, live request
| When it happens | Before the model is deployed, often once or periodically | Every single time a user asks a question or makes a request |
|---|---|---|
| Cost pattern | Large one-time or periodic cost | Smaller cost repeated per use, adds up at scale |
| Goal | Build the model's knowledge and patterns | Apply that knowledge to a real, live request |