Deriving an optimal noise adding mechanism for privacy-preserving machine learning

Authors Mohit Kumar
Michael Roßbory
Bernhard A. Moser
Bernhard Freudenthaler
Editors G. Anderst-Kotsis
A Min Tjoa
I. Khalil
et al.
Title Deriving an optimal noise adding mechanism for privacy-preserving machine learning
Booktitle Database and Expert Systems Applications - Proc DEXA 209 International Workshops
Type in proceedings
Publisher Springer
Series Communications in Computer and Information Science
Volume 1062
ISBN 978-3-030-27683-6
DOI 10.1007/978-3-030-27684-3_15
Month August
Year 2019
Pages 108-118
SCCH ID# 19019

Differential privacy is a standard mathematical framework to quantify the degree to which individual privacy in a statistical dataset is preserved.We derive an optimal (ǫ, δ)−differentially private noise adding mechanism for real-valued data matrices meant for the training of models by machine learning algorithms. The aim is to protect a machine learning algorithm from an adversary who seeks to gain an information about the data from algorithm’s output by perturbing the value in a sample of the training data. The fundamental issue of trade-off between privacy and utility is addressed by presenting a novel approach consisting of three steps: 1) the sufficient conditions on the probability density function of noise for (ǫ, δ)−differential privacy of a machine learning algorithm are derived; 2) the noise distribution that, for a given level of entropy, minimizes the expected noise magnitude is derived; 3) using entropy level as the design parameter, the optimal entropy level and the corresponding probability density function of the noise are derived. The derived optimal noise adding mechanism results in the magnitude of noise a multi-fold reduction (up to several tens times) over the classical Gaussian mechanism.