AI

An LLM agent may have access to a calculator, a browser, a code runner, GitHub, Slack, Jira, Google Drive, a database, and hundreds of internal APIs. Giving the agent more capabilities sounds useful. But there is a practical problem: every tool normally comes with a name, description, and input schema. If I place all of those definitions in every model request, the model has to read a large API catalog before it can answer a simple question. ...

Knowledge distillation transfers selected behavior from a larger teacher into a smaller student. The word distillation is doing too much work in today’s AI debate. In machine-learning research, it describes a family of methods for transferring knowledge from a teacher model to a student model. In product announcements, it often means training a smaller model from a larger model. In security discussions, the same word may describe collecting millions of responses from a competitor’s API. ...

AI coding agents are becoming more capable, but the way they connect to editors is still surprisingly fragmented. Imagine that an editor wants to support five coding agents. Without a shared protocol, the editor may need five custom integrations. Each integration has to handle prompts, streamed responses, file access, terminal output, tool calls, permissions, diffs, session history, and cancellation. The same problem exists on the other side. An agent developer who wants to support five editors may need to implement five different editor APIs. ...

DeepSeek recently open-sourced DeepSpec, a codebase for training and evaluating speculative decoding draft models, along with the DSpark paper and checkpoints. I wanted to understand what is actually new here, because speculative decoding itself is not new. The older idea is already powerful: use a small draft model to propose tokens, then let the large target model verify them. I explained that foundation in Speculative Decoding Explained. DSpark is interesting because it attacks two practical problems that show up when speculative decoding moves from a paper idea into production serving: ...

Speculative decoding is one of the most important tricks behind fast LLM serving. The basic idea is simple: Use a small fast model to guess several future tokens, then use the original large model to verify those guesses in one pass. If the guesses are right, the large model moves forward by multiple tokens instead of one. If a guess is wrong, the large model corrects it and the system continues. ...

The KV cache is one of those LLM inference terms that sounds simple until someone asks you to explain it from first principles. At a high level, the idea is: During generation, a transformer stores the previously computed key and value vectors, so it does not recompute them for the whole sequence every time it predicts the next token. That is the short answer. But the short answer hides the important part: why keys and values are reusable, why queries are not cached in the same way, and why this matters so much for latency. ...

Coding agents are useful when they can read code, edit files, run tests, and explain errors. But web development has a problem that does not fit neatly inside the file system: the real bug often lives in the browser. A React component may look fine in code but overflow on mobile. An API call may fail only after a specific login state. A button may be present in the DOM but not clickable. A performance issue may come from layout shifts, long tasks, font loading, image decoding, or network waterfalls. A console error may point to bundled JavaScript that needs source maps to be useful. ...

Originally published on DEV.to as a submission for the Google I/O Writing Challenge. At Google I/O 2026, the loud announcements were easy to spot: Gemini 3.5, Antigravity 2.0, Android agents, AI Studio upgrades, and a lot of new ways to build software with AI. The announcement I kept coming back to was much quieter: WebMCP. The Chrome docs describe it as a proposed open web standard that can be tested locally behind a Chrome flag and explored with demo apps. ...

When a production system fails, the hardest part is often not the fix. The hardest part is knowing where to look. That is the real value of observability. A service without observability feels like a black box. Requests go in, responses come out, and when something breaks we start guessing. With useful telemetry, that black box becomes closer to a glass box: we can see request paths, slow dependencies, errors, queueing, retries, model latency, token usage, and the exact step where a workflow fell apart. ...

When I evaluate an LLM system, one of the first latency metrics I look at is TTFT, or time to first token. This metric answers a simple question: After a user sends a request, how long does it take before the first output token appears? That sounds narrow, but it matters a lot. Users usually forgive a response that streams steadily after it starts. What feels bad is the dead time before anything appears on screen. ...