Latency

Latency is the delay between sending a query and the model's response - in practice, it's most often measured as time to first token. For smooth voice conversations or code autocompletion, fractions of a second matter; when generating a report, latency is less critical. Latency depends on model size, server load, prompt length, and operating mode (reasoning models "think" noticeably longer). App developers often juggle models: a fast one for instant reactions, a powerful one for tasks where users will wait.