Emerging edge AI systems call for device-level approaches that are inherently low-power, secure, and capable of managing uncertainty. In this talk, I will share our recent exploratory efforts toward realizing on-device probabilistic intelligence using custom-designed semiconductor devices. I will introduce Gaussian transistors that support analog Gaussian activation and probabilistic inference by harnessing device-level variability. These devices offer a potential path for implementing Bayesian operations directly at the transistor level. In parallel, we have been developing photo-spike photodetectors that convert light fluctuations into asynchronous spike trains, functioning as both neuromorphic input interfaces and entropy sources for physical randomness. While still in early stages, the combination of these platforms suggests a promising direction for hardware-embedded probabilistic learning, secure classification, and physical random number generation. This work aims to show how tuning the physics of emerging devices may open up new opportunities for edge AI systems.