A p-value in a classic t-test (arguably the most commonly used part of all of statistics) is the integral over Student's T-distribution. Basic linear regression is formulated in terms of the derivative of the loss function.
Read the Schaum's outlines for probability and statistics. They go a long way to helping sort yourself out. Also review the Khan academy lectures on the subject because probability is very counter-intuitive. If you know what central limit theorem is, what a t-test is and when to use it, and how to find a z score for a normally distributed thing then you have most of the major uses people have for stats. There is a ton more, but that covers the big honking things that most people are worried about.