FOLDS SEMINAR: The Hidden Width of Deep ResNets
/
Amy Gutmann Hall, Room 414
3333 Chestnut Street, Philadelphia, United States
Zoom link: https://upenn.zoom.us/j/6130182858 We present a mathematical framework to analyze the training dynamics of deep ResNets that rigorously captures practical architectures (including Transformers) trained from standard random initializations. Our approach combines stochastic approximation of ODEs with propagation-of-chaos arguments to obtain tight convergence rates to the “infinite size” limit of the dynamics. It yields the […]

