BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Penn Engineering Events - ECPv6.16.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Penn Engineering Events
X-ORIGINAL-URL:https://seasevents.nmsdev7.com
X-WR-CALDESC:Events for Penn Engineering Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20230312T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20231105T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20241111T110000
DTEND;TZID=America/New_York:20241111T120000
DTSTAMP:20260603T001355
CREATED:20241029T124653Z
LAST-MODIFIED:20241029T124653Z
UID:12502-1731322800-1731326400@seasevents.nmsdev7.com
SUMMARY:MEAM Ph.D. Thesis Defense: "Exploring Multimodal Sensing Across the Stack for Robot Manipulation"
DESCRIPTION:Despite substantial progress in robotics\, achieving human-like manipulation remains a significant challenge. Existing robotic systems typically leverage human-inspired sensory modalities: vision\, touch\, and proprioception. However\, these modalities are historically studied and integrated in isolation\, leading to limited performance in complex real-world tasks that require sensing across multiple modalities for robust generalization. As a result\, robots have struggled to transition from structured lab environments to effective real-world applications. \nThis persistent challenge highlights two critical limitations: the narrow focus on only human-inspired senses and the isolated integration of vision\, touch\, and proprioception fail to provide robots with the necessary adaptability for the real world. In contrast to traditional approaches focused on each of these three modalities\, complementary modalities and tightly integrated multimodal systems are underexplored. With the unprecedented availability of diverse off-the-shelf sensors\, powerful on-board computation to process rich data streams\, advances in data-driven control and perception frameworks\, and a new spotlight on robotic system integration\, we now face a unique opportunity to construct new multimodal sensing paradigms. \nThis thesis explores three complementary aspects of multimodality selection and integration across sensor design\, perception\, and RL control. First\, we address the challenge of integrating an additional modality without compromising existing functionality – a sensing mechanism design problem that often forces trade-offs between different sensing modes. We leverage a selectively transmissive membrane to enable proximity depth sensing that seamlessly augments the visuotactile modality. Next\, we exploit the complementary nature of our sensor’s dual modalities for sensor fusion. We demonstrate how carefully combining proximity and tactile modalities can enhance perception\, enabling more robust and informative contact patch detection. Finally\, we bridge the reality gap in robot policy learning\, where sim-to-real transfer is particularly challenging due to the complex physics of contact-rich manipulation. By developing a sim-to-real tactile skin model\, we achieve zero-shot transfer of tactile data during the in-hand translation task\, allowing us to evaluate the impact of combining tactile feedback with proprioception in this dexterous control task.
URL:https://seasevents.nmsdev7.com/event/meam-ph-d-thesis-defense-exploring-multimodal-sensing-across-the-stack-for-robot-manipulation/
LOCATION:Raisler Lounge (Room 225)\, Towne Building\, 220 South 33rd Street\, Philadelphia\, PA\, 19104\, United States
CATEGORIES:Doctoral,Dissertation or Thesis Defense
ORGANIZER;CN="Mechanical Engineering and Applied Mechanics":MAILTO:meam@seas.upenn.edu
END:VEVENT
END:VCALENDAR