Moment distances for comparing high-entropy

• Werner Zellinger
• Hamid Eghbal-zadeh
• Bernhard A. Moser
• Michael Zwick
• Edwin Lughofer
• Thomas Natschläger
• Susanne Saminger-Platz

Given two samples, the similarity of the distributions of the sample representations in the latent space of a discriminative model shall be enforced. Standard approaches are based on the minimization through probability metrics, e.g. by the Wasserstein metric, the Maximum Mean Discrepancy, or f-divergences. However, also moment distances not satisfying the identity of indiscernibles, i.e. pseudo-metrics, performed well in many practical tasks. The uniform distance between two distributions having finitely many moments in common can be very large.

The question is under which constraints on the distributions small values of moment distances imply distribution similarity. We show that the total variation distance between two distributions is small if the distributions are of high differential entropy constrained at finitely many moments that are similar for the two distribution. We also discuss existing relations between moment convergence and moment-constrained entropy convergence in the one-dimensional case. Our analysis leads to a new target error bound for domain adaptation, which is underpinned by numerical evaluations.