About this Event
135 N Skinker Blvd, St. Louis, MO 63112, USA
#SeminarSingle timescale actor critic: a small-gain analysis
Abstract: We consider the used-in-practice setting of actor-critic where proportional step-sizes are used for both the actor and the critic, with only one critic update with a single sample from the stationary distribution per actor step. Using a small-gain analysis, we prove convergence to a stationary point, with a sample complexity that improves the state of the art. The key technical challenge is in connecting the actor-critic to a perturbed gradient descent, which is often obtained by allowing for infinitely many critic steps and is not possible in single-time scale settings. This is a joint work with Alex Olshevsky at Boston University.