By Anik Kumar Paul, IISc Bengaluru

Abstract 

Stochastic Mirror Descent is an elegant optimization framework underlying many machine learning and reinforcement learning algorithms. In this talk, we revisit stochastic mirror descent through the lens of projected dynamical systems in a non-Euclidean geometry. This viewpoint provides a unified analytical approach that naturally accommodates non-convex and non-smooth optimization problems, while also allowing for iterate-dependent Markovian noise, thereby moving beyond the standard assumption of i.i.d. stochastic noise.  We present almost sure convergence,  finite-time concentration bounds, and show that the sample complexity guarantees under Markovian noise match the classical rates achieved by stochastic gradient methods under i.i.d. stochastic noise.  We then briefly extend this perspective to stochastic zeroth-order mirror descent. Finally,  we discuss how this perspective  extends to natural policy gradient methods, risk-sensitive reinforcement learning, and distributionally robust reinforcement learning, providing a unified theoretical foundation for modern reinforcement learning research.