BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Penn Engineering Events - ECPv6.16.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Penn Engineering Events
X-ORIGINAL-URL:https://seasevents.nmsdev7.com
X-WR-CALDESC:Events for Penn Engineering Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20251105T120000
DTEND;TZID=America/New_York:20251105T131500
DTSTAMP:20260601T205849
CREATED:20250821T204147Z
LAST-MODIFIED:20250821T204147Z
UID:20919-1762344000-1762348500@seasevents.nmsdev7.com
SUMMARY:ASSET Seminar: "The coverage principle in language models: From pre-training to test-time scaling"
DESCRIPTION:Test-time compute has emerged as a new axis for scaling language model capabilities\, yet we lack a principled understanding of this paradigm. What are the right algorithms and trade-offs for test-time scaling? What properties of the pre-trained model enable it? And can we better align pre-training recipes for test-time success? This talk addresses these questions through a unified lens of coverage. We first show that test-time scaling strategies like best-of-N sampling succeed if and only if the pre-trained model has coverage over high-quality responses. We then demonstrate that coverage\, and hence best-of-N performance\, can be improved through deliberate exploration\, either purely at test time or via RL-style post-training. Finally\, we ask why pre-training via next-token prediction yields models with good coverage in the first place. We uncover a rich theoretical landscape driven by an implicit bias of the next-token prediction objective\, while also identifying a fundamental misalignment between next-token prediction and coverage\, raising the possibility of future algorithmic innovations. \n  \nZoom: https://upenn.zoom.us/j/95189835192 \nPasscode: 797599
URL:https://seasevents.nmsdev7.com/event/asset-seminar-title-tbd-7/
LOCATION:Amy Gutmann Hall\, Room 414\, 3333 Chestnut Street\, Philadelphia\, 19104\, United States
CATEGORIES:Seminar
ORGANIZER;CN="AI-enabled Systems%3A Safe%2C Explainable%2C and Trustworthy (ASSET) Center":MAILTO:asset-info@seas.upenn.edu
END:VEVENT
END:VCALENDAR