Deriving an optimal noise adding mechanism for privacy-preserving machine learning

• Mohit Kumar
• Michael Roßbory
• Bernhard A. Moser
• Bernhard Freudenthaler

• G. Anderst-Kotsis
• o.Univ.Prof. Dipl.Ing. Dr. A Min Tjoa
• I. Khalil
• et al.

Differential privacy is a standard mathematical framework to quantify the degree to which individual privacy in a statistical dataset is preserved.We derive an optimal (ǫ, δ)−differentially private noise adding mechanism for real-valued data matrices meant for the training of models by machine learning algorithms. The aim is to protect a machine learning algorithm from an adversary who seeks to gain an information about the data from algorithm’s output by perturbing the value in a sample of the training data. The fundamental issue of trade-off between privacy and utility is addressed by presenting a novel approach consisting of three steps: 1) the sufficient conditions on the probability density function of noise for (ǫ, δ)−differential privacy of a machine learning algorithm are derived; 2) the noise distribution that, for a given level of entropy, minimizes the expected noise magnitude is derived; 3) using entropy level as the design parameter, the optimal entropy level and the corresponding probability density function of the noise are derived. The derived optimal noise adding mechanism results in the magnitude of noise a multi-fold reduction (up to several tens times) over the classical Gaussian mechanism.