BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Penn Engineering Events - ECPv6.16.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Penn Engineering Events
X-ORIGINAL-URL:https://seasevents.nmsdev7.com
X-WR-CALDESC:Events for Penn Engineering Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20230312T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20231105T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20241122T120000
DTEND;TZID=America/New_York:20241122T133000
DTSTAMP:20260602T214345
CREATED:20241122T135515Z
LAST-MODIFIED:20241122T135515Z
UID:12666-1732276800-1732282200@seasevents.nmsdev7.com
SUMMARY:ESE PhD Thesis Defense: "A Dynamical Systems Perspective on Optimization Algorithms"
DESCRIPTION:The intersection of machine learning (ML) and dynamical systems and control (S&C) has become a prominent area of research in recent years. While most work applies ML to S&C problems\, this dissertation explores the reverse direction: using S&C tools to address challenges in ML and optimization. This thesis is comprised of three parts: \nIn part I\, we examine connections between continuous-time dynamics and iterative optimization algorithms. Motivated by the matching rates observed by various optimization algorithms and their continuous-time counterparts\, we seek to gain a better understanding of the fundamental limits behind these matching rates. First\, we show that a simple rescaling of the gradient flow achieves finite-time stability\, which is clearly not preserved by its simplest discretization — the Forward Euler discretization — as it simply yields gradient descent. In fact\, this property is not exactly preserved by any explicit discretization as it relies on instantaneous corrections in the speed of the trajectories of the system\, which is not possible in discrete-time due to overshooting. Next\, we show that the rate of any continuous-time optimization system can be approximately preserved\, under sufficient regularity conditions\, by any sufficiently high order off-the-shelf ODE solver. \nIn part II\, we re-examine the relationship between convexity and smoothness and the role they play on the convergence rate of gradient-based optimization algorithms. Using and extending tools from part I\, we show that a rescaled form of gradient  descent achieves a rate that explicitly depends on given lower and upper bounds on the Bregman divergence of the cost function\, which collectively quantify convexity and smoothness. In particular\, we establish linear convergence with a generalized condition number\, generalizing the case of L-smooth and mu-strongly convex functions minimized via gradient descent. \nIn part III\, we discuss applications of S&C in two ML problems. The first is to re-examine the convergence of the expectation-maximization (EM) algorithm with a prior using Lyapunov stability theory. Next\, we consider the conformal training (ConfTr) method of Stutz et al (2022) and show that\, in its current form\, suffers from severe sample inefficiency. We propose a simple fix and provide a preliminary theoretical analysis using tools from linear systems theory. We also use the principles established in part II to guide the training procedure.
URL:https://seasevents.nmsdev7.com/event/ese-phd-thesis-defense-a-dynamical-systems-perspective-on-optimization-algorithms/
LOCATION:Zoom – Meeting ID: 946 6798 0381
CATEGORIES:Dissertation or Thesis Defense
ORGANIZER;CN="Electrical and Systems Engineering":MAILTO:eseevents@seas.upenn.edu
END:VEVENT
END:VCALENDAR