Zoom link:

https://upenn.zoom.us/j/98220304722

Abstract:

Modern ML models produce valuable predictions across various applications, influencing people’s lives, opportunities, and scientific advancements. However, these systems can fail in unexpected ways, generating unreliable inferences and perpetuating biases present in the data. These issues are particularly troubling in high-stakes applications, where models are trained on increasingly diverse, incomplete, and noisy data and then deployed in dynamic environments—conditions that often exacerbate test-time failures.

In response to these challenges, this talk explores a key question: How can fundamental statistical principles be harnessed to produce trustworthy predictive inference?

In the first part, I will present a new advancement in conformal prediction—a statistical wrapper for any black-box model that provides precise error bounds on ML predictions. I will focus on scenarios where training data is corrupted or biased, such as through missing features and labels, and introduce a framework for constructing predictive uncertainty estimates that remain valid despite distribution shifts between the available corrupted data and unknown clean data.

In the second part, I will show how sequential statistical testing can enable a novel test-time training scheme, allowing a pre-trained model to adapt online to unfamiliar environments. For instance, consider an image classification task where test images are captured under varying illumination conditions that differ from the training setup. Building on conformal betting martingales, I will first introduce a monitoring tool to detect data drifts. Using this tool, I will derive a rigorous ‘anti-drift correction’ mechanism grounded in (online) optimal transport principles. This mechanism forms the foundation of a self-training scheme that produces robust predictions invariant to dynamically changing environments.