We introduce the basic concepts of probability and apply them to simple physical systems and everyday life. We will discover the universal nature of the central limit theorem and the Gaussian distribution for the sum of a large number of random variables and discuss its relation to why thermodynamics is possible. Because of the importance of probability in many contexts and the relatively little time it will take us to consider more advanced topics, our discussion goes beyond what we will need for the applications of statistical mechanics. Computer simulations will be used to explore these concepts. These simulations can be found by searching ComPADRE for Open Source Physics, STP, or Statistical and Thermal Physics.