Latency
MLOps, LLMOps & DeploymentChapter 12: Going LiveTerm 208 of 253
Time to get a response. TTFT = time to first token.
At Sunrise School
Gopal asksParents wait 9 seconds for a reply.
Bit answersLatency is the time to answer. Streaming the first words in 0.8 seconds makes it feel instant.
In the real worldUser waits 1.2 s for first word.
Watch the short
Short coming soonSeason 3 · Day 24Follow @haigopala
Short coming soonSeason 3 · Day 24Follow @haigopala.telugu
Short coming soonSeason 3 · Day 24Follow @haigopala.hindi
Quick check
Quick check. Does streaming make the first words appear sooner? Yes or no?
Yes! Parents see the reply start right away.
Practise it
Try this term and all 253 others in the free Model Training Arena.
Open the Arena →Follow Bit daily: YouTube English · తెలుగు · हिंदी | Instagram English · తెలుగు · हिंदी