Speculative-Decoding

DeepSeek recently open-sourced DeepSpec, a codebase for training and evaluating speculative decoding draft models, along with the DSpark paper and checkpoints. I wanted to understand what is actually new here, because speculative decoding itself is not new. The older idea is already powerful: use a small draft model to propose tokens, then let the large target model verify them. I explained that foundation in Speculative Decoding Explained. DSpark is interesting because it attacks two practical problems that show up when speculative decoding moves from a paper idea into production serving: ...

Speculative decoding is one of the most important tricks behind fast LLM serving. The basic idea is simple: Use a small fast model to guess several future tokens, then use the original large model to verify those guesses in one pass. If the guesses are right, the large model moves forward by multiple tokens instead of one. If a guess is wrong, the large model corrects it and the system continues. ...