Large Language Models

Deferred Tools in LLMs: How Agents Use Thousands of Tools Without Loading Them All

An LLM agent may have access to a calculator, a browser, a code runner, GitHub, Slack, Jira, Google Drive, a database, and hundreds of internal APIs. Giving the agent more capabilities sounds useful. But there is a practical problem: every tool normally comes with a name, description, and input schema. If I place all of those definitions in every model request, the model has to read a large API catalog before it can answer a simple question. ...

Abstract illustration of DSpark routing confident drafted tokens through a verification scheduler

DSpark Explained: DeepSeek's Confidence-Scheduled Speculative Decoding

DeepSeek recently open-sourced DeepSpec, a codebase for training and evaluating speculative decoding draft models, along with the DSpark paper and checkpoints. I wanted to understand what is actually new here, because speculative decoding itself is not new. The older idea is already powerful: use a small draft model to propose tokens, then let the large target model verify them. I explained that foundation in Speculative Decoding Explained. DSpark is interesting because it attacks two practical problems that show up when speculative decoding moves from a paper idea into production serving: ...

Abstract illustration of a small draft model proposing tokens and a large target model verifying them

Speculative Decoding Explained: How LLMs Generate Faster Without Changing the Answer

Speculative decoding is one of the most important tricks behind fast LLM serving. The basic idea is simple: Use a small fast model to guess several future tokens, then use the original large model to verify those guesses in one pass. If the guesses are right, the large model moves forward by multiple tokens instead of one. If a guess is wrong, the large model corrects it and the system continues. ...

KV Cache Explained: Why LLMs Remember Keys and Values During Inference

The KV cache is one of those LLM inference terms that sounds simple until someone asks you to explain it from first principles. At a high level, the idea is: During generation, a transformer stores the previously computed key and value vectors, so it does not recompute them for the whole sequence every time it predicts the next token. That is the short answer. But the short answer hides the important part: why keys and values are reusable, why queries are not cached in the same way, and why this matters so much for latency. ...

RoPE Explained: The Positional Encoding Trick Behind Modern Language Models

When people talk about transformers, they usually focus on attention, scale, or training data. But one smaller design choice has an outsized effect on model quality: How does the model know where each token appears in the sequence? That question matters because transformers do not understand order by default. Without positional information, a sequence starts to look more like an unordered set of tokens than a structured sentence, paragraph, or program. ...

GPT-2 XL architecture diagram showing token embeddings, positional embeddings, 48 transformer blocks, 25 attention heads, and the output layer

Understanding LLM Architecture: Layers, Transformer Blocks, and Attention Heads

Large Language Models (LLMs) such as GPT-2, GPT-3, LLaMA, and BERT are built on top of the Transformer architecture. That architecture changed natural language processing by replacing recurrence with attention, which lets models process sequences more efficiently and capture long-range relationships more directly. If you are trying to understand what terms like layer, transformer block, and attention head actually mean, the easiest way is to follow the path a sentence takes through a GPT-style model. ...

How Much Do LLMs Hallucinate in Document Q&A? Key Lessons from a 172B-Token Study

If you are building a RAG system, internal knowledge assistant, or document search chatbot, one question matters more than almost anything else: When the answer is supposed to come from the provided documents, how often does the model still make things up? That is exactly what the March 9, 2026 paper “How Much Do LLMs Hallucinate in Document Q&A Scenarios? A 172-Billion-Token Study Across Temperatures, Context Lengths, and Hardware Platforms” tries to measure. ...