Penn Engineering Events

ASSET Seminar: "Wood Wide Models"
2024-10-09 12:00 - 13:15 (America/New_York) | Raisler Lounge (Room 225), Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, United States
Abstract:
Foundation models are monolithic models that are trained on a broad set of data, and which are then in principle fine-tuned to various specific tasks. But they are ill-suited to many heterogeneous settings, for instance numeric tabular data, or numeric time-series data, where training a single monolithic model over a large collection of such datasets is not meaningful. For instance, why should numeric times series of stock prices have anything to do with time series comprising the vital signs of an ICU patient? For such settings, we propose the class of wood wide models.
The wood wide web is often used to describe an underground network of fungal threads that connect many trees and plants together, which stands in contrast to a large concrete foundation on top of which we might build specialized buildings. Analogously, in contrast to a single foundation model upon which one might build specialized models, we can have many smaller wood wide models that all borrow subtler ingredients from each other. But to be able to share nutrients from the wood wide web, trees need a special root based architecture that can connect to these fungal threads. Accordingly, to operationalize wood wide models, we develop a novel neuro-symbolic architecture, that we term “neuro-causal”, that uses a synthesis of deep neural models and causal graphical models to automatically infer higher level symbolic information from lower level “raw features”, while also allowing for rich relationships among the symbolic variables. Neuro-causal models retain the flexibility of modern deep neural network architectures while simultaneously capturing statistical semantics such as identifiability and causality, which are important to discuss ideal, target representations and their tradeoffs. But most interestingly, these can further form a web of wood wide models when they borrow in part from a shared conceptual ontology, as well as causal mechanisms. We provide conditions under which this entire architecture can be recovered uniquely. We also discuss efficient algorithms and provide experiments illustrating the algorithms in practice.
Zoom Link (if unable to attend in-person): https://upenn.zoom.us/j/98029108883

Fall 2024 GRASP SFI: Tushar Kusnur, The AI Institute, “Robotic Information Gathering: Experiences and Perspectives”
2024-10-09 15:00 - 16:00 | Levine 307, 3330 Walnut Street, Philadelphia, PA, 19104, United States
This will be a hybrid event with in-person attendance in Levine 307 and virtual attendance on Zoom.
ABSTRACT
Despite all the interest and hype both from within and outside our community, robotics is a relatively new and highly specialized field. We embody substantial diversity in research direction and style, the focus and flavor of industrial companies, and academic departments and programs. There is no single way to “do” robotics. Similarly, the journeys of any two people in this field are unlikely to look the same. My work so far has shared a theme of “active information gathering” in academic research for novel approaches in specific contexts, as well as industry research that balances novel and practical systems in broader contexts. Consistent with that distinction, I will first talk about my past work at Carnegie Mellon including planning for robotic coverage and active sensing grounded in a system of multiple aerial vehicles. Next, I will briefly highlight some technical challenges relevant to my current work on the “Watch, Understand, Do” project at the AI Institute, grounded in mobile manipulation. I will share what I have learned about how different styles of research, collaboration, and project planning lend themselves to these different settings, with an eye on how we can make more informed decisions in our individual and collective journeys as roboticists.

CBE Seminar: "Organic Batteries for a More Sustainable Future" (Jodie Lutkenhaus, Texas A&M University)
2024-10-09 15:30 - 16:30 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
Abstract:
Cobalt, nickel, and lithium are essential ingredients in today’s lithium-ion batteries (LIBs), but their continued use presents economic, ethical, and environmental challenges. Society must now begin to consider the implications of a LIB’s full life cycle, including the carbon footprint, the economic and environmental costs, and material access. These challenges motivate the case for degradable or recyclable batteries sourced from earth-abundant materials whose life cycle bears minimal impact on the environment. This presentation considers organic polymer-based batteries, which have the potential to address many of these issues. Redox-active polymers form the positive and negative electrodes, storing charge through a reversible redox mechanism. We demonstrate polypeptide radical batteries that degrade on command into amino acids and by-products as a first step toward circular organic batteries. Further, we show the recycling of redox-active polymer electrodes using a solvent-based approach. Polymer-air batteries are examined as high-capacity alternatives to metal-air batteries. The molecular mechanism for each case is investigated, revealing pathways forward for improving each polymer’s performance. Taken together, organic batteries offer the promise of a circular platform free of critical elements.

MSE Seminar: "Color From Colorless Materials: Harnessing Multi-reflection Interference in Microstructures"
2024-10-10 10:30 - 12:00 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
Many of the colors found in nature, such as those of iridescent, color-shifting organisms like beetles, butterflies, and birds, are structural colors. Structural coloration is often generated by optical interference occurring within nanoscale periodic structures, like diffraction gratings, photonic crystals, or thin films. In these cases, the periodicity of the structure is similar to the wavelength of the visible light undergoing interference. However, I will describe the interesting observation and mechanism behind how optical interference and iridescent color can be generated by light interacting within much larger, microstructure structures. In this mechanism, light reflecting multiple times (such as by total internal reflection or a mirrored surface) and traveling along different paths within a microstructure can optically interfere. This effect happens in materials as simple as water droplets, but it can also be harnessed within far more complex 3D polymeric geometries to customize the interference. Ray tracing simulations coupled with color visualization and spectral analysis techniques can be used to model, examine, and rationalize the iridescence generated for a range of micro-geometries, including hemicylinders, hemispheres, truncated hemispheres, and other irregular structures under varying illumination conditions. Microstructure arrays patterned on surfaces with varying orientation and size lead to unique color-traveling optical effects and highlight opportunities for how multibounce reflection interference can be used to create customizable colored appearances. The findings provide a conceptual framework for rationalizing the multibounce interference mechanism and establish approaches for characterizing and tailoring the optical and iridescent properties of microstructured surfaces.

ESE Fall Seminar - "Deep Latent Variable Models for Compression and Natural Science"
2024-10-10 11:00 - 12:00 | Raisler Lounge (Room 225), Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, United States
Latent variable models have been an integral part of probabilistic machine learning, ranging from simple mixture models to variational autoencoders to powerful diffusion probabilistic models at the center of recent media attention. Perhaps less well-appreciated is the intimate connection between latent variable models and data compression, and the potential of these models for advancing natural science. This talk will explore these topics. I will begin by showcasing connections between variational methods and the theory and practice of neural data compression. On the applied side, variational methods lead to machine-learned compressors of data such as images and videos and offer principled techniques for enhancing their compression performance, as well as reducing their decoding complexity. On the theory side, variational methods also provide scalable bounds on the fundamental compressibility of real-world data, such as images and particle physics data. Lastly, I will also delve into applications, where I show how deep latent variable models allow solving challenging inverse problems in weather and climate modeling tasks.

BE Seminar: "Synthetic reconstitution of complex cellular behavior" (Ahmad Khalil, Boston University)
2024-10-10 15:30 - 16:30 | 216 Moore Building
Cells use genetically-encoded molecular circuits to execute diverse biological functions. We are developing novel tools of synthetic biology that allow us to construct regulatory circuitry inside living cells that recapitulate complex functions like those seen in nature. In this talk, I will describe how we use this approach to achieve three objectives. First, I will demonstrate how synthetic reconstitution provides a powerful way to understand fundamental principles of regulatory networks, which we have applied to guide discoveries in eukaryotic transcription regulation and epigenetics. Second, I will describe our advancements in developing synthetic circuit technologies that enable precise, instructive control of therapeutic human cell function to address challenges in emerging cellular therapies, such as CAR-T cells for cancer. Finally, I will share a future vision of how synthetic reconstitution can be used to engineer cells with little or no intrinsic therapeutic potential into powerful and scalable engines for generating custom, therapeutically-relevant molecules. Overall, by learning how to build biological systems from scratch, our broad goal is to connect the basic molecular building blocks of life to complex cellular behavior and ultimately to clinical applications.

CIS Seminar: "Neurosymbolic AI for Safety-Critical Agile Control"
2024-10-10 15:30 - 16:30 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
This talk overviews research at Caltech on designing hybrid or neurosymbolic AI systems that blend learning with symbolic structure, in order to achieve both the flexibility of the former and the formal interpretability and generalization power of the latter. By having systems that are formally interpretable, one can employ a wide range of formal analysis techniques to verify important properties of the overall system, such as those related to safety and stability, as well as use those analyses to guide system design and optimization. Focusing on formally interpretable structure arising from control and planning, I will present new algorithms and their deployment in a range of applications, including agile flight control under challenging and time-varying environments, controlling highly underactuated systems (e.g., one-legged hoppers), as well as briefly overview other related research.

Fall 2024 GRASP on Robotics: Antonio Loquercio, University of Pennsylvania, "Simulation: What made us intelligent will make our robots intelligent"
2024-10-11 10:30 - 11:45 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
This will be a hybrid event with in-person attendance in Wu and Chen and virtual attendance on Zoom.
ABSTRACT
Simulation-to-reality transfer is an emerging approach that enables robots to develop skills in simulated environments before applying them in the real world. This method has catalyzed numerous advancements in robotic learning, from locomotion to agile flight. In this talk, I will explore simulation-to-reality transfer through the lens of evolutionary biology, drawing intriguing parallels with the function of the mammalian neocortex. By reframing this technique in the context of biological evolution, we can uncover novel research questions and explore how simulation-to-reality transfer can evolve from an empirically driven process to a scientific discipline.

PICS Colloquium: Mean flow and turbulence in unsteady urban canopy flows
2024-10-11 14:00 - 15:00 | PICS Conference Room 534 – A Wing , 5th Floor, 3401 Walnut Street, Philadelphia, PA, 19104, United States
Advancing the current understanding and capability to predict atmospheric flow and related transport in urban areas is critical for many applications, including air quality modeling, urban climate, pedestrian comfort and structural resilience. Turbulence in these environments is rarely in equilibrium with the underlying surface and is typically characterized by strong departures from statistical stationarity. For example, the atmospheric boundary is often driven by a range of (sub)meso forcings that can evolve over sufficiently short time scales and result in unsteady flow conditions in cities. Yet, current theories describing mass, energy, and momentum transport in urban areas are largely established for equilibrium and statistically stationary flow. This presentation will provide an overview on the effects of a particular class of flow unsteadiness (flow pulsation) on the structure of mean flow and turbulence in urban areas. The discussion will focus on the impact of flow unsteadiness on flow statistics and on the geometry of coherent structures responsible for momentum and kinetic energy transport.

MEAM Seminar: "Propulsive Advantages of Coordinating Multiple Jets by Colonial Marine Organisms"
2024-10-15 10:15 - 11:15 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
Salps and siphonophores are widespread marine animals that occur in centimeters to meters-long colonial chains and employ multiple, pulsed swimming jets. We use a combination of approaches including in situ and lab experiments, flow visualizations and custom high-speed camera systems to understand the morphology, kinematics and fluid mechanics that underpin efficient movement in these colonial chains. Our results over the past decade show that: 1) kinematics drive effective fluid refill and maneuverability, 2) coordination among swimming units produces different swimming modes and enhances efficiency and, 3) diverse colony architectures give rise to a range of performance solutions. Colonial salps and siphonophores are a model system for understanding structure-fluid interactions in multi-jet locomotion. Our studies using 3D videography in the ocean will hopefully further our understanding of coordinated pulsed jets, and elucidate design principles optimized by nature.

ESE Fall Seminar - "Learning-NUM: Utility Maximization in Stochastic Queueing Networks"
2024-10-15 11:00 - 12:00 | Raisler Lounge (Room 225), Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, United States
We consider the problem of network utility maximization (NUM) and propose a new Learning-NUM framework, where the users’ utility functions are unknown apriori and the utility function values can be observed only after the corresponding traffic is delivered to the destination. We start by considering linear utility functions and propose a priority-based network control policy, that combines techniques from network control and multi-arm bandits to achieve logarithmic regret. We then consider the case of concave utility functions and design the Gradient Sampling Max-Weight algorithm (GSMW), based on the ideas of gradient estimation and Max-Weight scheduling, that achieves sublinear utility regret. We further demonstrate the applicability of the gradient sampling approach to minimum delay routing in wireless networks. Finally, we consider the general problem of reinforcement learning for queueing networks with unbounded state-spaces, with the goal of making control decisions that minimizing the queue length. We formulate the problem as an MDP, and propose a new reinforcement learning framework, called Truncated Upper Confidence Reinforcement Learning (TUCRL), that can achieve optimal performance. We show how this framework can be applied to deep reinforcement learning (DRL) for online stochastic network optimization.

CIS Seminar: "Optimal Oblivious Reconfigurable Networks"
2024-10-15 15:30 - 16:30 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
As Moore’s Law slows down, packet switch capabilities are falling behind datacenter demands. Recent hardware advances have enabled the new switching technology of nanosecond-scale rapid circuit switches. Combined with novel network designs, these have the potential to fully replace packet switches. This talk presents the Oblivious Reconfigurable Network (ORN) design paradigm which is ideally suited to this new switching technology. I describe how to design ORNs that work at datacenter scale, supporting tens of thousands of network nodes. And, I discuss an implementation, Shale, whose tradeoffs in latency and throughput are Pareto optimal among all ORN designs achieving orders of magnitude better latency and memory requirements than prior ORN designs at such scales.
A paper of this works appears in SIGCOMM 2024: “Shale: A Practical, Scalable Oblivious Reconfigural Networks”

ESE Guest Seminar - "Safe Offline RL for Constrained Markov Decision Process: Theory and Practice"
2024-10-16 11:00 - 12:00 | Greenberg Lounge (Room 114), Skirkanich Hall, 210 South 33rd Street, Philadelphia, PA, 19104, United States
Many constrained sequential decision-making processes such as safe AV navigation, wireless network control, caching, cloud computing, etc., can be cast as Constrained Markov Decision Processes (CMDP). Reinforcement Learning (RL) algorithms have been used to learn optimal policies for unknown unconstrained MDP. Extending these RL algorithms to unknown CMDP, brings the additional challenge of not only maximizing the reward but also satisfying the constraints. Further, in most of the practical applications, one has to rely on the offline database as online interaction might be costly or infeasible.
While the unconstrained offline RL setting is relatively well-understood, the offline CMDP or safe offline RL setup is not. For example, consider a database that consists of data coming from a safe behavioral policy, it remained an open problem on how to develop an algorithm that would provide safety while maximizing the reward with provable guarantee. In particular, the existing works on safe offline RL rely on the assumption that the database must contain state-action pairs coming from all the policies which is not practical in safety-critical setup as the database might not contain unsafe state-action pairs. We closed the gap in our recent research. In our work, we developed a weighted safe actor-critic (WSAC) algorithm that can produce a policy that outperforms any behavioral policy while maintaining the same level of safety, which is critical to designing a safe algorithm for offline RL. Additionally, we compare WSAC with existing state-of-the-art safe offline RL algorithms in several continuous control environments. WSAC outperforms all baselines across a range of tasks, supporting the theoretical results.

ASSET Seminar: "Some Displaced Vignettes on Generalized Notions of Equivariance"
2024-10-16 12:00 - 13:15 | Raisler Lounge (Room 225), Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, United States
Abstract:
The explicit incorporation of task-specific inductive biases through symmetry has emerged as a crucial design precept in the development of high-performance machine learning models. Symmetry-aware neural networks, such as group equivariant networks, have achieved notable success in areas like protein and drug design, where capturing task-specific symmetries improves generalization. Recent efforts have focused on models that relax equivariance, balancing flexibility and equivariance to enhance performance. In the first part of the talk, I will discuss the benefits of partial and approximate equivariance from a theoretical perspective, presenting quantitative bounds that demonstrate how models capturing task-specific symmetries lead to improved generalization. Utilizing this quantification, I will examine the more general question of dealing with approximate/partial symmetries and model mis-spefication, delineating conditions under which the model equivariance is optimal for a given level of data symmetry. In the second part, I will present a general formalism based on special structured matrices, which generalizes classical low-displacement rank theory of Kailath and co-workers, which can help in constructing approximately equivariant neural networks with significantly reduced parameter counts. In the last part, I will discuss some attempts at generalizing notions of equivariance in the context of language and compositional generalization. I will also talk about some ongoing work on using such notions for the problem of inverse protein folding.
Work done in collaboration with: Mircea Petrace (Pontificia Universidad Católica de Chile), Ashwin Samudre (Simon Fraser University), Brian D. Nord (Fermilab and University of Chicago), and Payel Das (IBM Research).
Zoom Link (if unable to attend in-person): https://upenn.zoom.us/j/96014696752

MEAM Ph.D. Thesis Defense: "Semantics-Driven Active Perception and Navigation with Aerial Robots"
2024-10-16 13:30 - 14:30 | Room 313, Singh Center for Nanotechnology, 3205 Walnut Street, Philadelphia, PA, 19104, United States
Autonomous aerial robots today are capable of safely navigating through cluttered, GPS-denied environments while constructing an accurate map that captures geometric features such as points, lines, and planes. Such maps are crucial for low-level planning and obstacle avoidance. However, beyond offering details on the density, layout, and dimensions of the environment, these maps provide limited information for semantically meaningful reasoning. For instance, they fall short in helping the robot identify where to find specific objects during search and rescue operations, which areas are relevant during infrastructure inspection or asset mapping, or in distinguishing between static and dynamic entities in the environment during localization and navigation. Such reasoning capabilities are especially important for resource-constrained robots, such as micro aerial vehicles, deployed in large-scale environments. In such scenarios, robots need to keep track of important, actionable information, while respecting their onboard computational and storage resource constraints. Sparse, semantically meaningful maps facilitate robust state estimation and storage-efficient mapping, while also enabling high-level reasoning that guides intelligent, task-relevant decision-making during navigation and exploration.
This thesis introduces a set of novel methodologies and algorithms for semantics-driven perception and autonomy, enabling robots to safely navigate and explore large-scale, complex environments while actively and collaboratively constructing high-quality, semantically meaningful maps. Specifically, we first present a monocular-camera-based semantic mapping system that integrates deep learning, visual tracking, and semantic Structure from Motion (SfM) for accurate fruit detection and mapping in orchards. We then develop a semantic Simultaneous Localization and Mapping (SLAM) integrated autonomous aerial navigation system for large-scale semantic mapping in under-forest-canopy environments. This system leverages real-time semantic SLAM for accurate pose estimation and timber metric assessment. Next, we introduce active metric-semantic SLAM systems for both urban outdoor and indoor environments. These systems use semantics to guide aerial robots to explore the environment and minimize both metric and semantic uncertainties. Lastly, bridging all efforts together, we propose a decentralized metric-semantic SLAM framework for autonomous navigation and exploration with heterogeneous robot teams operating across multiple environments. Extensive real-world experiments have validated the robustness and performance of the proposed methods across multiple aerial and ground robot platforms in various environments, including multi-floor indoor spaces, urban outdoors, forests, and orchards. Field deployments further showcase the system’s potential for direct application in solving important real-world problems, such as precision agriculture, forestry management, climate change mitigation, infrastructure inspection, and factory asset management. Finally, the proposed systems and algorithms are made available as open-source tools for public use.

Fall 2024 GRASP SFI: Jun-Yan Zhu, Carnegie Mellon University, "Ensuring Data Ownership in Generative Visual Models"
2024-10-16 15:00 - 16:00 | Levine 307, 3330 Walnut Street, Philadelphia, PA, 19104, United States
This will be a hybrid event with in-person attendance in Levine 307 and virtual attendance on Zoom.
ABSTRACT
Large-scale generative visual models have made content creation as little effort as writing a short text description. However, these models are typically trained on an enormous amount of Internet data, often containing copyrighted material, licensed images, and personal photos. How can we remove these images if creators decide to opt out? How can we properly compensate them if they choose to opt in?
In this talk, I will first describe an efficient method for removing copyrighted materials, artistic styles of living artists, and memorized images from pretrained text-to-image models. I will then discuss our data attribution algorithms for assessing the influence of each training image for a generated sample. Collectively, we aim to enable creators to retain control over the ownership of training images.

CBE Seminar: "Optimizing Work and Heat flows in Sustainable Chemical Processes Using Attainable Regions" (Diane Hildebrandt, Rutgers University)
2024-10-16 15:30 - 16:30 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
Abstract:
A chemical process transforms feed materials into desired products, necessitating the exchange of heat and work between the process and the environment. Optimizing process performance requires careful selection of chemical pathways, process parameters, and heat/workflows. The reversible process sets the upper-performance limit for sustainable design, thereby setting a target for process design. We propose using a plot of enthalpy (H) against Gibbs Free Energy (G) to represent, synthesize, and analyze heat and workflows in chemical processes. This graphical approach is particularly useful for identifying and assessing process reversibility. We define the Thermodynamic Attainable Region (AR T ) as the set of all (delta)H and (delta)G across all possible processes that convert a given feed(s) into specified products. In this talk, we will present a methodology for calculating the AR T . By leveraging the visual representation of the AR T , we can systematically analyze and compare the inherent heat and workflows of different processes, feeds, and products. We will consider some simple examples to illustrate the impact of the selection of chemical pathways, process parameters, and feeds on process heat and workflows and reversibility.

MSE Seminar “Shaping a Soft Future"
2024-10-17 10:30 - 12:00 | Wu & Chen Auditorium
Existing devices—such as cell phones, computers, and robots – are made from rigid materials, which is in direct contrast to the soft materials that compose the human body. In this talk, I will discuss several topics related to studying and harnessing soft materials within the context of creating devices with tissue like properties.

Liquid metal: Gallium-based liquid metals are often overlooked despite their remarkable properties: melting points below room temperature, water-like viscosity, low-toxicity, and effectively zero vapor pressure (they do not evaporate). They also have, by far, the largest interfacial tension of any liquid at room temperature. Normally small volumes of liquids with large tension form spherical or hemi-spherical structures to minimize surface energy. Yet, these liquid metals can be patterned into non-spherical shapes (cones, wires, antennas) due to a thin, oxide skin that forms rapidly on its surface.
Shape reconfiguration: Perhaps the most fascinating aspect of liquid metals it the ability to use interfacial electrochemistry chemistry to remove / deposit the oxide to manipulate the surface tension of the metal over unprecedented ranges (from the largest tension of any known liquid to near zero!). This allows manipulating the shape and position of the metal for shape reconfigurable devices.
Lonogels: Soft materials that are tough (that is, they do not readily tear or fail mechanically) are important for a number of applications, including encapsulation of devices. Recently, we discovered a simple way to create ultra-tough ionogels, which are polymer networks swollen with ionic liquids. These materials are tougher than cartilage and compatible with 3D printing.

This work has implications for soft and stretchable electronics; that is, devices with desirable mechanical properties for human-machine interfacing, soft robotics, and wearable electronics.

IDEAS Seminar: "Equivariant Neural Inertial Odometry"
2024-10-17 11:00 - 12:00 | Room 401B, 3401 Walnut, 3401 Walnut Street, Philadelphia, PA, 19104, United States
Abstract:
In this talk, we introduce a new class of problems related to integrating inertial measurements obtained from an IMU that play a significant role in navigation combined with visual data. While there have been tremendous technological advances in the precision of instrumentation, integrating acceleration and angular velocity still suffers from drift in the displacement estimates. Neural networks have come to the rescue in estimating displacement and the associated uncertainty covariance. However, such networks do not consider the physical roto-reflective symmetries inherent in IMU data, leading to the need to memorize the same priors for every possible motion direction, which hinders generalization. In this work, we characterize these symmetries and show that the IMU data and the resulting displacement and covariance transform equivariantly when rotated around the gravity vector and reflected with respect to arbitrary
planes parallel to gravity. We propose a network for predicting an equivariant gravity-aligned frame from equivariant vectors and invariant scalars derived from IMU data, leveraging expressive linear and non-linear layers tailored to commute with the underlying symmetry transformation. Such a canonical frame can precede existing architectures that are end-to-end or filter-based. We will include an introduction to the inertial filtering problem and we will present
results in real-world datasets.
Zoom link: https://caltech.zoom.us/j/88141815847?pwd=[REDACTED]

Fall 2024 GRASP Seminar: Bob Collins & Yanxi Liu, Penn State University, “From Vision to Dynamics”
2024-10-17 14:00 - 15:00 | Levine 307, 3330 Walnut Street, Philadelphia, PA, 19104, United States
*This seminar will be held in-person in Levine 307 as well as virtually via Zoom.
ABSTRACT
In the realms of health and sports, quantitative analysis of human movement provides guidance for individualized performance characterization, training, and health assessment. Data collection for biomechanical analysis of human motion is typically performed using expensive, specialized hardware that requires experiments to occur in a constrained lab setting. Computer vision algorithms using inexpensive, off-the-shelf video cameras for data collection would enable observation in more natural environments with minimal cost. Towards this end, this talk reports on:
• Our effort in collecting the first publicly available Taiji-MultiModal (PSU TMM100) dataset containing 100 sequences of simplified 24-form Taiji with synchronized mocap, video, and foot-insole pressure maps.
• An unsupervised learning method for training a light-weight encoder suitable for 3D body pose classification and sequence-to-sequence temporal alignment.
• The first deep learning baseline that demonstrates reliable and repeatable mapping from either a single frame or short sequence of human pose (kinematics) into predicted foot pressure map output (dynamics), leading to image-based stability monitoring in natural environments.

Responsible Innovation
2024-10-17 15:00 - 19:00 | Penn Washington Center, 101 Constitution Ave. NW, Washington DC, DC, 20001, United States
Rapid advancements in healthcare, technology, finance and beyond present significant opportunities and challenges. As we look to develop leading-edge solutions to provide a better future for all, Penn Engineering and Perry World House are committed to promoting responsible innovation practices.
Join us for an inspiring afternoon as we launch the Responsible Innovation initiative at the Penn Washington Center in Washington, DC. This event will bring together leaders in engineering, academia, industry, and policy for an engaging dialogue on responsibly shaping the future of innovation.
Agenda:
3:00: Guests arrive
3:15: Opening remarks by Vijay Kumar, Nemirovsky Family Dean, Penn Engineering
3:25: Remarks by Ted Schlein, partner at Kleiner Perkins, Executive Chairman and Founding Partner at Ballistic Ventures, and former chairman of the National Venture Capital Association (NVCA)
3:40: Keynote address by Michael Horowitz, Richard Perry Professor and Director of the Perry World House at the University of Pennsylvania, and former Deputy Assistant Secretary of Defense for Force Development and Emerging Capabilities
4:20: Faculty panel discussion moderated by Susan Davidson, Weiss Professor of Computer and Information Science (CIS), featuring:

Michael Kearns, National Center Professor of Management & Technology, CIS
Dan Koditschek, Alfred Fitler Moore Professor, Electrical and Systems Engineering (ESE), CIS, and Mechanical Engineering and Applied Mechanics (MEAM)
Mark Yim, Asa Whitney Professor of Mechanical Engineering, MEAM, CIS, and ESE

5:00: Presentation by George Pappas, UPS Foundation Professor of Transportation, ESE, CIS, and MEAM
5:30: Cocktail Hour
Don’t miss this opportunity to engage with thought leaders and be part of Penn Engineering’s bold vision for advancing responsible innovation.
Please make sure to RSVP by Friday, October 11, to secure your spot.

BE Seminar: "DNA microscopy in two and three dimensions" (Joshua Weinstein, University of Chicago)
2024-10-17 15:30 - 16:30 | 216 Moore Building
Lymphatic, nervous, and tumoral tissues, among others, exhibit physiology that emerges from three-dimensional interactions between genetically unique cells. Technologies capable of volumetrically imaging transcriptomes, genotypes, and morphologies in a single de novo measurement can provide a critical lens into the biological complexity of living systems. We present experimental and computational work to develop DNA microscopy: a modality of imaging that captures physical images of specimen genetic content using a massive distributed network of DNA molecules inside it. We demonstrate DNA microscopy in two-dimensional cell cultures and genome-wide in intact zebrafish embryos.

Fall 2024 GRASP on Robotics: Zak Kassas, IEEE AESS DL & The Ohio State University, "Ad Astra: Navigation with Megaconstellation LEO Satellites"
2024-10-18 10:30 - 11:45 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
This will be a hybrid event with in-person attendance in Wu and Chen and virtual attendance on Zoom.
ABSTRACT
We are witnessing a space renaissance. Tens of thousands of broadband low Earth orbit (LEO) satellites are expected to be launched by the end of this decade. These planned megaconstellations of LEO satellites along with existing constellations will shower the Earth with a plethora of signals of opportunity, diverse in frequency and direction. These signals could be exploited for positioning, navigation, and timing (PNT) in the inevitable event that global navigation satellite system (GNSS) signals become unavailable (e.g., in deep urban canyons, under dense foliage, during unintentional interference, and intentional jamming) or untrustworthy (e.g., under malicious spoofing attacks).
This talk will overview the challenges associated with exploiting megaconstellation LEO satellites for PNT purposes, namely their unknown signals, poorly known ephemerides, loose synchronization and oscillator instability, and propagation effects. Next, a framework termed STAN: simultaneous tracking and navigation will be introduced to overcome these challenges. We will present an end-to-end approach, spanning theoretical modeling and analysis, specialized cognitive software-defined radio (SDR) design, practical PNT algorithms, and experimental demonstrations of STAN on the ground and aerial vehicles, navigating with multi-constellation LEO satellite signals (Starlink, OneWeb, Orbcomm, Iridium, and NOAA) to an unprecedented level of accuracy. Insights into future research directions and engineering implementation challenges will be provided as concluding remarks.

MEAM Seminar: "Engineering Mechanics of Architected Hard-Soft Composites: Experiment, Simulation, and Theory"
2024-10-22 10:15 - 11:15 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
Enhancing the resistance of human-made brittle materials to fracture is challenging due to the limited microstructural toughening mechanisms. This seminar makes a case for engineering toughening mechanisms in brittle materials by developing purposeful architected arrangements of material inspired by natural systems. Experimental fracture mechanics in hard-soft (cementitious-elastomeric) ‘Nacre-like’ composites based on the tabulated brick-and-mortar arrangement of mollusk shells is presented. Tablet sliding and soft interlayer energy dissipation are among the nacre’s hierarchical toughening mechanisms, leading to its significantly higher fracture toughness than its major brittle constituent (~95%, aragonite). Here, it is hypothesized that tablet sliding and tortuous crack propagation (crack deflection and crack bridging) are the key mechanisms that promote inelastic deformation and increase the size of the fracture process zone in brittle material. These mechanisms significantly enhance fracture toughness and ductility by an order of magnitude compared to constituent hardened cement paste.
To better understand crack propagation in hard-soft composites a unified large-deformation constitutive framework was developed (implemented via user-element subroutine within the finite element software Abaqus). Interfacial properties play a crucial role in the fracture process. The proposed computational framework couples the phase-field approach for bulk fracture with a potential-based cohesive zone model (CZM) to study crack propagation in multi-material (e.g., hard-soft, hard-hard) containing an interface. The phase-field captures crack initiation and propagation in the bulk constituents, and CZM (PPR) captures the role of the interface failure (e.g., delamination, deflection). The framework’s validation against linear elastic fracture mechanics theory for hard-hard composites with an interface is discussed, including the competition between crack penetration into the bulk vs. crack deflection into the interface. The framework is a numerical tool for probing or designing architected hard-soft materials with enhanced performance characteristics and mechanisms. The seminar extends the design of architected materials beyond hard-soft composites using statistical mechanics. The degree of ‘order’ in material(s) arrangement can be quantified using proposed translational or orientational order parameters (T, Q) from perfectly ordered to disordered and ideal random. Proper quantification of disorder in contrast to other approaches (Voronoi tessellation or perturbation methods) allows for probing the disorder-property relationship. Combined with advanced manufacturing techniques (robotic additive, laser processing, etc.), disorder can be used as a new way to design engineering materials and structures.

BE Doctoral Dissertation Defense: "Data Driven Approaches for Optimizing Antiseizure Medication Management in Epilepsy" (Nina Ghosn)
2024-10-22 12:00 - 14:00 | BRB Auditorium
Update 10/14/24: Please note the new time and location for Nina Ghosn’s Doctoral Dissertation Defense below.

The Department of Bioengineering at the University of Pennsylvania and Dr. Brian Litt are pleased to announce the Doctoral Dissertation Defense of Nina Ghosn.

Title: DATA DRIVEN APPROACHES FOR OPTIMIZING ANTISEIZURE MEDICATION MANAGEMENT IN EPILEPSY

Advisor: Brian Litt

Date: October 22nd, 2024

TIme: 12:00 PM

Location: Biomedical Research Building (BRB) auditorium.

Zoom link: https://upenn.zoom.us/j/98684417609

The public is welcome to attend.

CIS Seminar: "How to Design Useful and Usable AI-Powered Applications"
2024-10-22 15:30 - 16:30 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
AI-powered applications are exciting because of their potential to support people in unprecedented ways but they are also particularly challenging to design right: How does one design a useful AI-enabled product if the underlying AI is occasionally wrong? How can we make these applications feel predictable even though the AI technology is complex and can appear unpredictable? While some specialized design knowledge related to Human-AI Interaction already exists, the production of this knowledge is not keeping up with the pace at which new AI-powered applications are invented. Consequently, without much fanfare or deliberation (or recognition of the fact!), some critical knowledge gaps are getting filled with reasonable-sounding but unverified assumptions. I will present a series of experiments (related to predictive text entry and AI-supported decision making) demonstrating that several of the key assumptions, upon which a lot of research projects and products rest, are wrong. I will then describe recent projects that build on corrected knowledge foundations and share some early promising results. I conclude with two calls to action for our field. First, we need to engage in critical technical practice, i.e., explicitly name, assess and correct (if necessary) the hidden assumptions of our field. Second, with Human-AI Interaction being a relatively new field but one that many people depend on, we need a greater investment in systematic production, synthesis and dissemination of reliable design knowledge for Human-AI Interaction.

ASSET Seminar: "Towards Sustainable Artificial Intelligence and Datacenters"
2024-10-23 12:00 - 13:15 | Raisler Lounge (Room 225), Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, United States
Abstract:
As the impact of artificial intelligence (AI) continues to proliferate, computer architects must assess and mitigate its environmental impact. This talk will survey strategies for reducing the carbon footprint of AI computation and datacenter infrastructure, drawing on data and experiences from industrial, hyperscale systems. First, we analyze the embodied and operational carbon implications of super-linear AI growth. Second, we re-think datacenter infrastructure and define a solution space for carbon-free computation with renewable energy, utility-scale batteries, and job scheduling. Finally, we develop strategies for datacenter demand response, incentivizing both batch and real-time workloads to modulate power usage in ways that reflect their performance costs. In summary, the talk provides a broad perspective on sustainable computing and outlines the many remaining directions for future work.
Zoom Link (if unable to attend in-person): https://upenn.zoom.us/j/92591691189

Fall 2024 GRASP SFI: Carl Vondrick, Columbia University, “Making Sense of the Multimodal World”
2024-10-23 15:00 - 16:00 | Levine 307, 3330 Walnut Street, Philadelphia, PA, 19104, United States
This will be a hybrid event with in-person attendance in Levine 307 and virtual attendance on Zoom.
ABSTRACT
People experience the world through modalities of sight, sound, words, touch, and more. By leveraging their natural relationships and developing multimodal learning methods, my research creates artificial perception systems with diverse skills, including spatial, physical, logical, and cognitive abilities, for flexibly analyzing visual data. This multimodal approach provides versatile representations for tasks like 3D reconstruction, visual question answering, and object recognition, while offering inherent explainability and excellent zero-shot generalization across tasks. By closely integrating diverse modalities, we can overcome key challenges in machine learning and enable new capabilities for computer vision, especially for the many upcoming applications where physical interaction are required.

CBE Seminar: "Backbone Modifications in Peptide Natural Products" (James Link, Princeton University)
2024-10-23 15:30 - 16:30 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
Abstract:
The posttranslational modifications (PTMs) in RiPPs (ribosomally synthesized and post-translationally modified peptides) dictate their 3D structure and their bioactivity. An underlying structural feature of many RiPPs is macrocyclization, installed by a growing number of different enzymatic strategies. Another common class of PTM in RiPPs is backbone modification, such as the formation of thiazol(in)es and oxazol(in)es. Our group has recently been interested in aspartimidylation, a backbone modification occurring at Asp residues that installs a metastable succinimide moiety into several different RiPPs. This talk will focus on recent work on the O-methyltransferase enzymes that install aspartimide in lasso peptides, graspetides, and a new class of peptides, the imiditides. The surprising stability of these aspartimide moieties will also be discussed within a kinetic framework. Finally, the talk will discuss potential chemical and biological ramifications of the aspartimide moiety in RiPPs.

MSE Seminar: "Kagome Metals and Their Unusual Electronic Properties"
2024-10-24 10:30 - 12:00 | Wu and Chen Auditorium (Room 101), Levine Hall, 3330 Walnut Street, Philadelphia, PA, 19104, United States
In this talk, I will discuss new inroads in the study of electronic order within classes of metals built from kagome lattices or networks of corner sharing triangles. The electronic band structures of these compounds are known to host a series of features such as Dirac crossings, saddle points, and flat bands at select carrier fillings. Tuning the electron filing about these features has the potential to stabilize a variety of exotic electronic states such as orbital magnetism, bond density wave order, and unconventional superconductivity; however experimental realization of these states has been a historical challenge. Recently, a number of new compounds built from kagome lattices with band fillings near each of these features have been discovered, and I will provide an overview of progress in studying their anomalous properties. Particular focus will be given to electronic instabilities realized in kagome metals with their Fermi levels close to the saddle points in their band structures.
