Stochastic stability analysis of perturbed learning automata with constant step-size in strategic-form games

Authors Georgios C. Chasparis
Editors
Title Stochastic stability analysis of perturbed learning automata with constant step-size in strategic-form games
Booktitle Proceedings of the 2017 American Control Conference
Type in proceedings
Publisher IEEE
ISBN 978-1-5090-5994-2
Month May
Year 2017
Pages 4607-4612
SCCH ID# 16053
Abstract

This paper considers a class of reinforcement-learning that belongs to the family of Learning Automata and provides a stochastic-stability analysis in strategic-form games. For this class of dynamics, convergence to pure Nash equilibria has been demonstrated only for the fine class of potential games. Prior work primarily provides convergence properties of the dynamics through stochastic approximations, where the asymptotic behavior can be associated with the limit points of an ordinary-differential equation (ODE). However, analyzing global convergence through the ODE-approximation requires the existence of a Lyapunov or a potential function, which naturally restricts the applicability of these algorithms to a fine class of games. To overcome these limitations, this paper introduces an alternative framework for analyzing stochastic-stability that is based upon an explicit characterization of the (unique) invariant probability measure of the induced Markov chain.