Advanced defensive distillation with ensemble voting and noisy logits

In this paper we provide an approach for deep neural networks that protects against adversarial examples in image classification-type problems. blackUnlike adversarial training, our approach is independent to the obtained adversarial examples through min-max optimization. The approach relies on the defensive distillation mechanism. This defence mechanism, while very successful at the time, was defeated in less than a year due to a major intrinsic vulnerability: the availability of the neural network’s logit layer to the attacker. We overcome this vulnerability and enhance defensive distillation by two mechanisms: 1) a mechanism to hide the logit layer (noisy logit) which increases robustness at the expense of accuracy, and, 2) a mechanism that improves accuracy but does not always increase robustness (ensemble network). We show that by combining the two mechanisms and incorporating a voting method, we can provide protection against adversarial examples while retaining accuracy. We formulate potential attacks on our approach with different threat models. The experimental results demonstrate the effectiveness of our approach. We also provide a robustness guarantee along with an interpretation for the guarantee.