BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Penn Engineering Events - ECPv6.15.18//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Penn Engineering Events
X-ORIGINAL-URL:https://seasevents.nmsdev7.com
X-WR-CALDESC:Events for Penn Engineering Events
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20210314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20211107T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20220313T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20221106T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20230312T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20231105T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20220413T150000
DTEND;TZID=America/New_York:20220413T160000
DTSTAMP:20260406T013306
CREATED:20220408T195819Z
LAST-MODIFIED:20220408T195819Z
UID:6714-1649862000-1649865600@seasevents.nmsdev7.com
SUMMARY:Spring 2022 GRASP SFI: Georgios Georgakis\, University of Pennsylvania\, “Cross-modal Map Learning for Vision and Language Navigation”
DESCRIPTION:*This will be a HYBRID Event with in-person attendance in Levine 512 and Virtual attendance via Zoom \nWe consider the problem of Vision-and-Language Navigation (VLN) in previously unseen realistic indoor environments. Arguably\, the biggest challenge in VLN is grounding the natural language to the visual input. The majority of current methods for VLN are trained end-to-end using either unstructured memory such as LSTM\, or using cross-modal attention over the egocentric RGB-D observations of the agent. We are motivated by studies on navigation of biological systems that suggest humans build cognitive maps during such tasks. In contrast to other works\, we argue that an egocentric map offers a more natural representation for this task. In this talk\, we will explore a novel navigation system for the VLN task in continuous environments that learns a language-informed representation for both map and trajectory prediction. This approach semantically grounds the language through an egocentric map prediction task that learns to hallucinate information outside the field-of-view of the agent. This is followed by spatial grounding of the instruction by path prediction on the egocentric map. We experimentally test the basic hypothesis that language-driven navigation can be solved given a map\, and then show competitive results on the full VLN-CE benchmark.
URL:https://seasevents.nmsdev7.com/event/spring-2022-grasp-sfi-georgios-georgakis-university-of-pennsylvania-cross-modal-map-learning-for-vision-and-language-navigation/
LOCATION:Levine 512
ORGANIZER;CN="General Robotics%2C Automation%2C Sensing and Perception (GRASP) Lab":MAILTO:grasplab@seas.upenn.edu
END:VEVENT
END:VCALENDAR