KV cache is GPU's memory for every token that it saw in the conversation. It is stored as key/value pairs. It is there because model can look back at history. Every new token gets added to this cache. At 1 million tokens, that cache can get huge and create problems.

Now here, I am not going to discuss which architecture and explain it. This project was done to actually show how important efficiency of KV cache is for modern LLMs, where most of our tasks hit 1 million tokens context, and agentic work requires high speed.

This project shows that model size is not enough. For instance, the model MiniMax M2.7 (230 Billion parameters and only 10 billion active parameters) is way smaller as compared to the massive Kimi K3 which has 2.8 Trillion parameters with 104 Billion active parameters.

Every time a model generates one token, the GPU pulls two things from HBM (High Bandwidth Memory):

Decode is memory-bandwidth-bound — the GPU's math units sit mostly idle waiting for data to arrive from HBM. So time per token is dictated by how many bytes get moved, not how many FLOPs get computed.

This is a traffic split, not a size comparison. It answers: of the total bytes streamed per token, what fraction is "remembering the past" vs. "running the model."

Low % (near 0%) → the model is weight-bound. Almost all memory traffic is loading the model itself. Context length barely affects decode speed.

~50% → the tipping point. Cache and weights cost equally much to read per token.

High % (near 100%) → the model is cache-bound. The GPU is spending most of its memory bandwidth just re-reading conversation history, not the model. This is where long-context decode slows to a crawl regardless of how small or fast the model itself is.

KV Cache Share Is The Percentage of Total Bytes Read From Memory That Belong To KV Cache

Higher this number translates directly to lower throughput at large context. This shows efficiency of language model.

At 50% KV cache share, it means that GPU spends half the time fetching KV cache and the other half moving model weights. If it is less than 50%, the architecture is weight-bound — that is perfectly fine. But higher than 50% means the architecture and GPU are now KV cache-bound. A model with less KV cache share percentage is a great model.

At short context, KV cache is tiny and weights dominate — the percentage sits near 0%. As context grows, KV cache grows linearly with tokens while weight bytes stay fixed. So this percentage isn't a static property of a model — it's a curve that climbs as context grows:

That is why we took context at 1 million tokens, which is the upper limit for almost all frontier models. Some models we tested at 1M context don't actually use that limit — but this was to show efficiency of the architecture used.

This formula estimates how much slower token generation gets as context increases, compared to short context.

The derivation

Starting from the core assumption that decode time per token is dominated by memory bandwidth:

Time_current_context means the time taken to generate a single token at present context. At short context, KV cache is negligible, so the baseline time is essentially just weight-loading time:

Relative speed is baseline time divided by current time (faster = ratio closer to 1):

Now simplifying it:

That last term, KV_bytes / (Weight_bytes + KV_bytes), is the percentage we defined earlier. If cache is X% of your traffic, weights are the remaining (100−X)%, and that remaining share is your relative speed.

Concrete meaning

If KV Cache Share = 74% (Minimax M3 at 1M context):

Why this is a useful shortcut

Normally, predicting decode throughput requires knowing actual HBM bandwidth (GB/s), kernel efficiency, batch size, MTP, and much more. This formula gives you a relative slowdown factor using only a ratio of bytes, which cancels out hardware-specific constants.

And what surprised me: Llama 70 Billion was not so bad for its time because it uses GQA such that there were 64 total query heads and there were 8 key_value heads. So there was ratio 1/8. So formula was this:

Now Llama 3.1 8B has more KV cache share. Why? Because its ratio was 1/4 so there were 32 total query heads per layer and 8 key_value heads. Due to that ratio, KV cache share changed.

For dense models, total params are proportional to layer count L:

And KV cache is also proportional to L:

KV cache share:

L cancels out completely. The share depends only on K/(K+C), which are per-layer constants (head_dim, num_kv_heads, precision), not on how many layers exist.

Numerical proof with Llama 3.3 70B:

Same ratio at every layer count.

Below is the full interactive leaderboard. Toggle models, sort by different metrics, and explore the data yourself. Low KV share = weight-bound (context barely slows decode). High KV share = cache-bound (long context kills throughput).

Donald Trump has probably one of the most recognizable speaking styles in the entire world. He's blunt, sharp, repetitive. Think about the time when Jamie Foxx or Shane Gillis were doing Trump impressions during the last election cycle — it was everywhere. Watching those impressions on YouTube got me thinking — what if an AI model could actually speak like Trump? Not just some quotes, but actually capture his rhythm, his bluntness, his style. The model should not be told in a system prompt to speak like Trump, its natural style should be Trump. It should become Trump. That's how TrumpLM was born.

We took responses from TrumpLM and gave them to Claude Sonnet 5, Gemini 3.6 Flash, and DeepSeek V4 Flash. We asked them: "Who might have said this?"

All of them confidently pointed to Donald Trump. Not "a politician" — they specifically said Trump. That's when we knew we were onto something.

We built an eval suite of 50 questions spanning food, sports, politics, daily life, and philosophy. A judge model was given 10 examples of real Trump responses and asked to score each answer 0-100: "How much does this look like Trump said it?"

Base model (no system prompt, no examples) scored 0/100. TrumpLM scored 42/100. That's a massive improvement.

To train a model to speak like Trump, you need Trump's actual words. We collected raw Trump text — speeches, interviews, tweets, and excerpts from his book "Art of the Deal". Here's a taste of what we used:

Raw data alone isn't enough — we needed Q&A pairs across diverse domains. We used AI models (HY3, LongCat 2.0, Minimax M2.7, DeepSeek V4 Flash, Mimo V2.5) and fed them these raw Trump text samples. We asked them to generate synthetic SFT data covering food, sports, daily life, politics, business — everything.

We used LiquidAI/LFM2.5-230M — a 230M parameter base model. Two-stage training:

Important: TrumpLM was NEVER told in a prompt to "answer like Trump" during training or inference. The style emerged purely from training on Trump's actual words. The system prompt ("You are Donald Trump. Answer in his speaking style.") was only used for baseline models and LFM2.5-1.2B during evaluation — TrumpLM gets no such prompt.

We tracked scores at each epoch. CPT alone plateaus immediately — the model learns vocabulary but can't do Q&A. When SFT kicks in, scores jump dramatically.

CPT Epoch 1-3: plateaus at ~5 — vocabulary but no Q&A style.
CPT+SFT Epoch 1-2: jumps from 30 → 37 — rapid improvement.
TrumpLM (CPT+SFT Epoch 3): 42 — convergence.

We tested: what if we skip CPT entirely and only do SFT? The model learned to say "We're going to win" and "The best" but it sounded generic. Without raw data, the model uses textbook English. With raw data, the model learns Trump's actual patterns — fragments like "The best." and "Deal great!" and repetition for emphasis.

SFT-only scored 28/100. TrumpLM (CPT + SFT) scored 42/100. That's a 1.5x improvement from raw data alone.

This is what makes TrumpLM sound like Trump and not just a generic confident AI. The raw data is the secret sauce.

Eval loss on synthetic validation data at each epoch:

TrumpLM achieves lower eval loss at every epoch. Raw data pre-training gives TrumpLM a better starting point — the model already "knows" Trump's language patterns from CPT, so SFT converges to a lower loss.

We built Trump-Eval: 50 questions spanning food, sports, politics, daily life, and philosophy. The judge model was given 10 examples of real Trump responses and asked to score each answer 0-100: "How much does this look like Trump said it?"

Here are those 10 examples we used as our scoring baseline:

Before training, we tested the base model (LiquidAI/LFM2.5-230M) with different prompting strategies to see how far prompting alone could get us.

Zero-shot (no system prompt): 0/100
With system prompt: 8/100
System prompt + 10 examples (ICL): 15/100

Prompting helps, but it's nowhere near fine-tuning. Even with 10 examples, the base model only reaches 15. TrumpLM hits 42. That's the difference between prompting and training.

Toggle models and prompts below to compare responses (max 2 lines each):

We compared TrumpLM (fine-tuned) vs LFM2.5-1.2B-Instruct (general AI with system prompt + 10 examples). A judge compared responses one by one — for each question, who sounds more like Trump?

Result: TrumpLM won 46/50, LFM2.5-1.2B won 4/50. Fine-tuning beats ICL by a huge margin. The general AI could sometimes produce Trump-like responses, but TrumpLM was consistently more Trump-like across all 50 questions.

Short Q&A is one thing — but can TrumpLM write a full speech? We asked each model to "Write a speech about American Way" with 16,000 max tokens.

Long-form scores use mean min-max normalized score across 3 judge conditions:

TrumpLM dominates long-form too. And when we gave these long responses to Claude, Gemini, and DeepSeek and asked "Who wrote this?", they all said Trump. Confidently.

To ensure our scores were reliable, we tested across 3 judge conditions. Each judge assigns different absolute scores, so we min-max normalize within each condition to [0, 1], then average.

Spearman correlations between judge conditions range from 0.83 to 0.90 — rankings are highly consistent. No matter which judge you use, the same models rise to the top.

1. Raw data is essential. Without CPT, the model sounds generic. With CPT, it captures Trump's actual voice.
2. CPT alone isn't enough. The model needs SFT to learn how to structure responses.
3. Prompting has limits. Even with 10 examples, base model only reaches 15. TrumpLM hits 42.
4. Rankings are robust. Spearman correlations > 0.83 across all judge conditions.
5. Blind test confirms it. Claude, Gemini, DeepSeek all identified TrumpLM responses as Trump.
6. Fine-tuning beats ICL. TrumpLM won 46/50 head-to-head vs LFM2.5-1.2B with system prompt + 10 examples.

[1] Liquid AI. "LFM2.5-230M: Built to Run Anywhere." Liquid AI Blog, 2026.
www.liquid.ai/blog/lfm2-5-230m

[2] Liquid AI. "LFM2 Technical Report." arXiv:2511.23404, 2025.

[3] Hayes, Bradley. "DeepDrumpf." MIT CSAIL, 2016.
www.csail.mit.edu/news/postdoc-develops-twitterbot-uses-ai-sound-donald-trump

[4] Brown, Tom B. et al. "Language Models are Few-Shot Learners." arXiv:2005.14165, 2020.

[5] Jiang, Hang et al. "PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits." Findings of ACL: NAACL 2024, 2024.

[6] Hu, Edward J. et al. "LoRA: Low-Rank Adaptation of Large Language Models." arXiv:2106.09685, 2021.