FOLDS seminar: Weak to Strong Generalization in Random Feature Models
/
Amy Gutmann Hall, Room 414
3333 Chestnut Street, Philadelphia, United States
Zoom link: https://upenn.zoom.us/j/98220304722 Weak-to-Strong Generalization (Burns et al., 2023) is the phenomenon whereby a strong student, say GPT-4, learns a task from a weak teacher, say GPT-2, and ends up significantly outperforming the teacher. We show that this phenomenon does not require a strong and complex learner like GPT-4, nor pre-training. We consider students and […]

