In the rapidly evolving domain of Earth Sciences, the application of statistics and machine learning (ML) has become indispensable for advancing research and understanding complex phenomena. With the increasing availability of large, high-quality datasets, researchers are harnessing the power of these methodologies to analyze, interpret, and model diverse Earth Science problems. However, the choice of which technique to apply depends on the specific problem, data type, and desired outcome. This article aims to provide a comprehensive guide to the foundations of applied statistics and ML in Earth Sciences, illustrating the most effective techniques for various research problems and helping researchers make informed decisions on where to use what.