Llm-Training

Knowledge distillation transfers selected behavior from a larger teacher into a smaller student. The word distillation is doing too much work in today’s AI debate. In machine-learning research, it describes a family of methods for transferring knowledge from a teacher model to a student model. In product announcements, it often means training a smaller model from a larger model. In security discussions, the same word may describe collecting millions of responses from a competitor’s API. ...