As the adoption of large-scale learning for control continues to grow, developing sample-efficient algorithms has become critical. Yet, even in simple settings, algorithms achieving optimal sample complexity for specific problem instances often remain unknown. Motivated by this limitation, we discuss recent progress toward understanding sample-efficient methods in learning-enabled control. We first examine the statistical limits of offline reinforcement learning with continuous state, action, and observation spaces by deriving lower bounds on the cost of a learned controller that characterize inherently challenging problem instances. We then introduce efficient algorithms and establish tight finite-sample bounds on the cost they incur for controlling a general class of nonlinear dynamical systems. These results underscore the critical role of dataset quality and motivate our subsequent exploration of optimal task-oriented experiment design. Finally, we consider large-scale pre-trained models for control, analyzing how models trained across diverse tasks can be fine-tuned for new control objectives with limited data. We approach this problem through the lens of representation learning in adaptive control and provide upper bounds on the incurred regret.