Machine learning is transforming numerous aspects of modern society, and its expanding use in high-stakes applications calls for responsible development. In this talk, I will present my research on the foundations and methodologies for building trustworthy ML, centered on three interconnected challenges: generalization, memorization, and privacy. First, I will show how information-theoretic tools can be used to analyze generalization across different learning setups. Next, I will describe my work on the fundamental limits of memorization in certain high-dimensional convex settings, showing a precise trade-off between memorization and accuracy. Finally, I will propose adaptive and efficient optimization algorithms under differential privacy—a well-established framework designed to protect sensitive data and limit memorization risk—that adapt to the properties of the dataset, resulting in smaller error. My results highlight how these three pillars interact, and I will conclude by outlining my plans for future research.