hAIgopala › AI terms › Rapid Fire (3 terms per short) › Day 14

Max Tokens

Generative AI & LLMsChapter 4: Choosing the LLMTerm 63 of 253

Limit on how long the model's output can be.

At Sunrise School

Gopal asks

Some replies are far too long for WhatsApp.

Bit answers

Set max tokens to 150. I keep answers short enough to read on a phone, and it saves money.

In the real world

Cap responses to 500 tokens.

Watch the short

Short coming soonSeason 5 · Day 14Follow @haigopala

Quick check

Quick check. For fee answers, do we want lots of variety? Yes or no?

Taught together with

Top-p / Top-k Sampling · Multi-Head Attention

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी