BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Penn Engineering Events - ECPv6.16.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Penn Engineering Events
X-ORIGINAL-URL:https://seasevents.nmsdev7.com
X-WR-CALDESC:Events for Penn Engineering Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20251023T120000
DTEND;TZID=America/New_York:20251023T130000
DTSTAMP:20260601T221316
CREATED:20250828T202806Z
LAST-MODIFIED:20250828T202806Z
UID:20942-1761220800-1761224400@seasevents.nmsdev7.com
SUMMARY:FOLDS seminar: An Information Geometric Understanding of Deep Learning
DESCRIPTION:Zoom link: https://upenn.zoom.us/j/98220304722 \n  \nI will argue that properties of natural data are what predominantly\nmake deep networks so effective. To that end\, I will show that deep\nnetworks work well because of a characteristic structure in the space\nof learnable tasks. The input correlation matrix for typical tasks has\na “sloppy” eigenspectrum where eigenvalues decay linearly on a\nlogarithmic scale. As a consequence\, the Hessian and the Fisher\nInformation Matrix of a trained network also have a sloppy\neigenspectrum. Using this idea\, I will demonstrate an analytical\,\nnon-vacuous PAC-Bayes bound on the generalization error for general\ndeep networks. \nI will show that the training process in deep learning explores a\nremarkably low dimensional manifold\, as low as three. Networks with a\nwide variety of architectures\, sizes\, optimization and regularization\nmethods lie on the same manifold. Networks being trained on different\ntasks (e.g.\, different subsets of ImageNet) using different methods\n(e.g.\, supervised\, transfer\, meta\, semi and self-supervised learning)\nalso lie on the same low-dimensional manifold. \nI will show that typical tasks are highly redundant functions of their\ninputs. Many perception tasks\, from visual recognition\, semantic\nsegmentation\, optical flow\, depth estimation\, to vocalization\ndiscrimination\, can be predicted extremely well regardless of whether\ndata is projected in the principal subspace where it varies the most\,\nsome intermediate subspace with moderate variability—or the bottom\nsubspace where data varies the least. \nReferences\n1. Does the data induce capacity control in deep learning? Rubing\nYang\, Jialin Mao\, and Pratik Chaudhari. [ICML ’22]\nhttps://urldefense.com/v3/__https://arxiv.org/abs/2110.14163__;!!IBzWLUs!Tq9FM96P-1mf3aRxklnZ7t8aLcjOIeWQz7icW_vh7HTMTDM2izgvjEC74IXkk0qZ7_TO9jbK-CF-J1f8wzalGOTVcA$\n2. The Training Process of Many Deep Networks Explores the Same\nLow-Dimensional Manifold. Jialin Mao\, Itay Griniasty\, Han Kheng Teoh\,\nRahul Ramesh\, Rubing Yang\, Mark K. Transtrum\, James P. Sethna\, Pratik\nChaudhari. [PNAS 2024]. https://urldefense.com/v3/__https://arxiv.org/abs/2305.01604__;!!IBzWLUs!Tq9FM96P-1mf3aRxklnZ7t8aLcjOIeWQz7icW_vh7HTMTDM2izgvjEC74IXkk0qZ7_TO9jbK-CF-J1f8wzYgaqrIWg$\n3. Many Perception Tasks are Highly Redundant Functions of their Input\nData. Rahul Ramesh\, Anthony Bisulco\, Ronald W. DiTullio\, Linran Wei\,\nVijay Balasubramanian\, Kostas Daniilidis\, Pratik Chaudhari.\n(in submission) https://urldefense.com/v3/__https://arxiv.org/abs/2407.13841__;!!IBzWLUs!Tq9FM96P-1mf3aRxklnZ7t8aLcjOIeWQz7icW_vh7HTMTDM2izgvjEC74IXkk0qZ7_TO9jbK-CF-J1f8wzaKl_77LQ$\n4. An Analytical Characterization of Sloppiness in Neural Networks:\nInsights from Linear Models. Jialin Mao\, Itay Griniasty\, Yan Sun\, Mark\nK Transtrum\, James P Sethna\, Pratik Chaudhari.\n(under review) https://urldefense.com/v3/__https://arxiv.org/abs/2505.08915__;!!IBzWLUs!Tq9FM96P-1mf3aRxklnZ7t8aLcjOIeWQz7icW_vh7HTMTDM2izgvjEC74IXkk0qZ7_TO9jbK-CF-J1f8wzYMqke2wg$
URL:https://seasevents.nmsdev7.com/event/folds-seminar-tba-5/
LOCATION:Amy Gutmann Hall\, Room 414\, 3333 Chestnut Street\, Philadelphia\, 19104\, United States
CATEGORIES:Seminar,Colloquium
END:VEVENT
END:VCALENDAR