Training vs Inference
Training is the process of teaching a model by feeding it large amounts of data, inference is the process of actually using that already-trained model to produce an answer. Training happens once, or periodically, inference happens every time someone uses the model.
Training
The process of feeding data to an AI model to help it learn patterns, adjust its internal mathematical weights, and make accurate predictions.
Inference
The process of using a trained AI model to make predictions, generate content, or solve tasks based on new, unseen input data. Its speed drives user experience.
Before the model is deployed, often once or periodically
Every single time a user asks a question or makes a request
Large one-time or periodic cost
Smaller cost repeated per use, adds up at scale
Build the model's knowledge and patterns
Apply that knowledge to a real, live request
| When it happens | Before the model is deployed, often once or periodically | Every single time a user asks a question or makes a request |
|---|---|---|
| Cost pattern | Large one-time or periodic cost | Smaller cost repeated per use, adds up at scale |
| Goal | Build the model's knowledge and patterns | Apply that knowledge to a real, live request |