Distilling the Knowledge in a Neural Network

distillation has been around for 12 years. it's not new in terms of ML techniques. https://arxiv.org/abs/1503.02531 although i doubt there has been a legal case over it yet in the context of the legality of stealing shit but IANAL.

This is exactly what the DeepSeek team did, and now Anthropic is repackaging it a year later, calling it “subliminal learning” or using the teacher and student analogy to take credit for work done by Chinese researchers. What? Distillation is way older. The Hinton paper was from 2015 (maybe there is even earlier work): https://arxiv.org/abs/1503.02531 When I was still in academia, we were distilling models from BERT/RoBERTa-large to smaller models (remember when those models were considered large?) in 2019 using logits and L2 distance of hidden layers. Before that we were also doing distillation of our own transformer/lstm models on model outputs (though with a different motivation than model compression, to learn selectional preferences, etc.).

Distillation (as you may be aware). https://arxiv.org/abs/1503.02531 Having to do that in each instance is still really cumbersome for cheap mass deployment compared to just making a digital-style exact copy, but then again I guess a main argument for wanting these systems is that they'd be doing things unachievable in practice on digital computers. In some cases one might be able to distill to digital arithmetic after the heavy parts of the optimization are done, for replication, distribution, better access for software analysis, etc.

> You're teaching your model to copy an other model's outputs. That's a time honoured tradition in ML, invented by the father of the field himself, Geoffrey Hinton, in 2015. > Distilling the Knowledge in a Neural Network https://arxiv.org/abs/1503.02531

Bit of a tangent -- I am not sure about how our brains deal with ambiguity vs sharp boundaries, but we can see a neural network change its tune quite a bit by adjusting this single number (i.e. the temperature parameter of Softmax function). This trick has been used for knowledge distillation (KD) as well. Hinton et. al. wrote this quite approachable paper on the topic: https://arxiv.org/pdf/1503.02531.pdf Turns out, one _needs_ nuance when one is trying to do KD. I am not sure if nuance and ambiguity as you describe are the same thing. But, even though it does not _feel_ like it, good generalizations over the lifetime of an organism might require dealing with nuance appropriately. A subject matter expert might have more to say about this. I cannot imagine that this hasn't been researched in humans.

Check out this work from Rich Caruana & collaborators on model compression: http://www.cs.cornell.edu/~caruana/compression.kdd06.pdf which was a precursor to the model distallation work from Geoff Hinton: https://arxiv.org/abs/1503.02531

https://arxiv.org/abs/1711.09784 https://arxiv.org/abs/1708.01785 https://arxiv.org/abs/1710.00935 https://arxiv.org/abs/1702.04595 https://arxiv.org/abs/1801.01693 https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1712.03781 https://arxiv.org/abs/1710.07535 https://arxiv.org/abs/1702.02540 https://arxiv.org/abs/1603.02518 http://ieeexplore.ieee.org/document/977279/ etc It is kind of far cry from established practice but how far cry from established implementation are your fears of unintended abuse? It's not nearly at the point yet to be paranoid. Having bugs in ANN before debugging makes them no different from usual software that is already used in forensic research. And nobody in their right mind will deploy networks that impossible or prohibitively difficult to debug for any kind of critical tasks. Otherwise irrational people can abuse power even without any software or ANNs.

I saw a talk on this paper a couple years ago. https://arxiv.org/abs/1503.02531 The method is to train a smaller model on the predictions of a large model or ensemble. I'd be interested in knowing other techniques as well.

Sorry, but this comment is completely uniformed. Explaining ML inference is an active area of research, and there are plenty of decent approaches which are getting good results. For example, LIME[1] can explain why a neural network doing visual classification came out with the result that it did. Distilling the Knowledge in a Neural Network[2] is another approach which leads towards simpler representations. LIME isn't perfect, but that's the nature of research (and why DARPA funds this kind of thing). this is a fool's errand "Distilling the Knowledge in a Neural Network", authors: Geoffrey Hinton, Oriol Vinyals, Jeff Dean Fools, all of them. DARPA should ask HN next time it wants to know about the current state of AI. [1] https://github.com/marcotcr/lime [2] https://arxiv.org/abs/1503.02531

Adding perturbations is a common technique in machine learning. For example, the evolution strategies approach [https://blog.openai.com/evolution-strategies/] makes 100s of randomly tweaked policies, evaluates them against the task, and recombines them with weights proportional to their performance. Another approach is to train several different neural networks (an ensemble) on a task, and then train a final neural network based on the average of the ensemble. [https://arxiv.org/pdf/1503.02531v1.pdf] So you can probably replicate the useful features of flawed copying between humans.

Well for starters, fine-tuning can be done in a variety of different ways. You can pretrain your model with a larger, different dataset, or you can train an autoencoder that learns some useful representation of that larger dataset and use the encoder as a base for fine tuning. Another approach I've seen that was really cool is Model Distillation [0], which is basically the training of a smaller NN with the inputs and outputs of a larger NN (where the output is slightly modified to increase gradients and make training faster). [0] https://arxiv.org/pdf/1503.02531v1.pdf

https://arxiv.org/abs/1503.02531 Modern applications of small networks regularly reduce sizes from larger state-of-the-art networks using distillation. Distillation compacts neural networks while affecting accuracy minimally. Instead of pruning directly from the large network, just learn how it generalizes. Takes fewer nodes / overall operations (Multiplications / Additions).

That's already a known method to transfer "knowledge" from one model to another. I should double-check before quoting a paper, but I think that this one talks about this (http://arxiv.org/abs/1503.02531). You train many models. Then you "distill" their predictions into one model by using the multiple predictions (from many models) as targets (for the single model trained afterwards). You're right to point out that humans don't do that. I think it would be "cheating" if you train BetaGo on AlphaGo, for the purposes for doing that experiment. The goal would be to have some kind of "clean room" where people fumble around. Of course, you can also run the other experiment to see how fast you can bootstrap BetaGo from AlphaGo. That's also interesting.

I see no mention of it, but I'd be surprised if they didn't use some form of knowledge distilling [1] (which Hinton came up with, so really no excuse), to condense a large neural network into a much smaller one. [1] http://arxiv.org/abs/1503.02531

A possibly relevant research paper that they didn't mention: "Distilling the Knowledge in a Neural Network" http://arxiv.org/abs/1503.02531