Scalable and efficient machine learning (ML) systems have been instrumental in fueling recent advancements in ML capabilities. However, further scaling these systems requires more than simply increasing the number and performance of accelerators. This is because modern ML deployments rely on complex pipelines composed of many diverse and interconnected systems. 

In this talk, I will emphasize the importance of building scalable systems across the entire ML pipeline. In particular, I will explore how large-scale ML training pipelines, including those deployed at Meta, require distributed data storage and ingestion systems to manage massive training datasets. Optimizing these data systems is essential as data demands continue to grow. To achieve this, I will demonstrate how synergistic optimizations across the training data pipeline can unlock performance and efficiency gains beyond what isolated system optimizations can achieve. While these synergistic optimizations are critical, deploying them requires navigating a large system design space. To address this challenge, I will next introduce cedar, a framework that automates the optimization and orchestration of ML data processing for diverse training workloads. Finally, I will discuss further opportunities in advancing the scalability, security, and capabilities of the hardware and software systems that continue to drive increasingly sophisticated ML training and inference pipelines.