ABSTRACT:

In recent years, foundation models—large pretrained models that can be adapted for a wide range of tasks—have achieved state-of-the-art performance on a variety of tasks. While the pretrained models are trained on broad data, the adaptation (or fine-tuning) process is often performed on limited data. As a result, the challenges of distribution shift, where a model is deployed on a different distribution as the fine-tuning data remain, albeit in a different form.

First, via experiments on pretrained vision and language models, we show different kinds of “catastrophic forgetting’’ where pretrained information is forgotten and correspondences between and in-distribution and out-of-distribution features are weakened. As a result, fine-tuned models are not maximally robust to distribution shifts. We then provide new fine-tuning and prompting methods, backed by theoretical insights, that minimize such distortion and vastly improve accuracy and robustness. On the flip side, our work shows that pretrained knowledge can be hard to get rid of, thereby underlining the potential perils of overreliance on fine-tuning for safety.