BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Penn Engineering Events - ECPv6.16.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Penn Engineering Events
X-ORIGINAL-URL:https://seasevents.nmsdev7.com
X-WR-CALDESC:Events for Penn Engineering Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20250328T093000
DTEND;TZID=America/New_York:20250328T103000
DTSTAMP:20260602T125146
CREATED:20250327T203006Z
LAST-MODIFIED:20250327T203006Z
UID:13702-1743154200-1743157800@seasevents.nmsdev7.com
SUMMARY:CIS Seminar: "Pareto-efficient AI systems: Expanding the quality and efficiency frontier of AI"
DESCRIPTION:We have made exciting progress in AI by massive models on massive amounts of data center compute. However\, the demands for AI are rapidly expanding. I identify how to maximize performance under any compute constraint\, expanding the Pareto frontier of AI capabilities.\n \nThis talk builds up to an efficient language model architecture that expands the Pareto-frontier between quality and throughput efficiency. In motivation\, the Transformer\, AI’s current workhorse architecture\, is memory hungry\, severely limiting its throughput\, or amount of text it can process per second. This has led to a Cambrian explosion of alternate efficient architecture candidates proposed across prior work. Prior work has painted an exciting picture: there exists architectures that are asymptotically faster than Transformers\, while also matching quality. However\, I ask\, if we’re using asymptotically faster building blocks\, are we giving something up in quality?\n\n\nIn part one\, we build understanding. Indeed\, there’s no free lunch! I present my work to identify and explain the fundamental quality and efficiency tradeoffs between different classes of architectures. Methods I developed for this analysis are now ubiquitous in the development of language models.\n\n\nIn part two\, we measure how AI architecture candidates fare on the tradeoff space. A major hurdle\, however\, is that we lack implementations of the architectures that that run at peak-efficiency on modern hardware. Further\, many proposed architectures are asymptotically fast\, but not wall-clock fast. I present ThunderKittens\, a new programming library I built to help AI researchers write simple\, hardware-efficient algorithms across hardware platforms.\n\n\nIn part three\, we expand the Pareto-frontier of the tradeoff space. I present the BASED architecture\, which is built from simple\, hardware-efficient components. I released the state-of-the-art 8B-405B Transformer-free language models\, per standard evaluations\, all on an academic budget.\n\n\nGiven the massive investment into language models\, this work has had significant impact and adoption in research\, open-source\, and industry.
URL:https://seasevents.nmsdev7.com/event/cis-seminar-pareto-efficient-ai-systems-expanding-the-quality-and-efficiency-frontier-of-ai-2/
LOCATION:Amy Gutmann Hall\, Room 414\, 3333 Chestnut Street\, Philadelphia\, 19104\, United States
ORGANIZER;CN="Computer and Information Science":MAILTO:cherylh@cis.upenn.edu
END:VEVENT
END:VCALENDAR