hAIgopala › AI terms › Bit at Work › Day 24

Latency

MLOps, LLMOps & DeploymentChapter 12: Going LiveTerm 208 of 253

Time to get a response. TTFT = time to first token.

At Sunrise School

Gopal asks

Parents wait 9 seconds for a reply.

Bit answers

Latency is the time to answer. Streaming the first words in 0.8 seconds makes it feel instant.

In the real world

User waits 1.2 s for first word.

Watch the short

Short coming soonSeason 3 · Day 24Follow @haigopala

Quick check

Quick check. Does streaming make the first words appear sooner? Yes or no?

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी