We revisit the “One Learning Algorithm Hypothesis” of Andrew Ng (Google Brain) according to which the brain of higher-level animals and of humans processes and perceives sensory data (vision, sound, haptics) with the same abstract algorithmic architecture. We develop models, based on our earlier work on automatic target recognition with radar and other sensors, face recognition and image classification, which employ a multi-resolution preprocessor, followed by a group-invariance based feature extractor, followed by a machine learning (ML) module that employs the two fundamental algorithms of Kohonen Learning Vector Quantization (LVQ), for supervised learning, and Self-Organizing Map (SOM), for unsupervised learning. In addition the model and algorithms utilize a “global” feedback from the output of the overall system back to the feature extractor and to the multiresolution preprocessor. We first summarize briefly our older results with such algorithms and their remarkable, domain agnostic, performance on various applications. We then provide our recent results on the mathematical analysis of the resulting Tree Structure Learning Vector Quantization (TSLVQ) ML architecture and algorithms. We introduce and integrate Deterministic Annealing (DA) with our older architecture and demonstrate the resulting tremendous reduction in data required for learning and application. The new algorithms allow even on-line progressive learning. We utilize Bregman divergences as dissimilarity measures, which allows us to provide direct transition from “dissimilarity distance” to probability of error, which cannot be achieved with the commonly used metric-based dissimilarity measures. We show that many deep learning network architectures can be mapped to this “universal” architecture. We show that the integrated algorithm converges to the true Bayes decision surface, albeit with variable resolution at various parts of it, as required. The latter brings a tight connection to integrated hypothesis testing with compressed data. We demonstrate the results in various applications and close with future directions and extensions.