Inference

Inference is the phase when a trained model is used: it receives your query and generates a response, token by token. This distinguishes everyday model work from training, which happened once, earlier, on massive data. AI service costs are mainly inference costs - that's why providers optimize with quantization, better hardware, and architectures like MoE. When a chat "writes" a response live, you're watching inference in real time, usually in streaming mode.