Description“This textbook is a well-rounded, rigorous, and informative work presenting the mathematics behind modern machine learning techniques. It hits all the right notes: the choice of topics is up-to-date and perfect for a course on data science for mathematics students at the advanced undergraduate or early graduate level. This book fills a sorely-needed gap in the existing literature by not sacrificing depth for breadth, presenting proofs of major theorems and subsequent derivations, as well as providing a copious amount of Python code. I only wish a book like this had been around when I first began my journey!” -Nicholas Hoell, University of Toronto“This is a well-written book that provides a deeper dive into data-scientific methods than many introductory texts. The writing is clear, and the text logically builds up regularization, classification, and decision trees. Compared to its probable competitors, it carves out a unique niche. -Adam Loy, Carleton CollegeThe purpose of Data Science and Machine Learning: Mathematical and Statistical Methods is to provide an accessible, yet comprehensive textbook intended for students interested in gaining a better understanding of the mathematics and statistics that underpin the rich variety of ideas and machine learning algorithms in data science.Key FeaturesFocuses on mathematical understanding.Presentation is self-contained, accessible, and comprehensive.Extensive list of exercises and worked-out examples.Many concrete algorithms with Python code.Full color throughout.Table of ContentsPreface Notation1. Importing, Summarizing, and Visualizing DataIntroductionStructuring Features According to TypeSummary TablesSummary StatisticsVisualizing DataPlotting Qualitative VariablesPlotting Quantitative VariablesData Visualization in a Bivariate Setting Exercises2. Statistical LearningIntroductionSupervised and Unsupervised LearningTraining and Test LossTradeoffs in Statistical LearningEstimating RiskIn-Sample RiskCross-ValidationModeling DataMultivariate Normal ModelsNormal Linear ModelsBayesian Learning Exercises3. Monte Carlo MethodsIntroductionMonte Carlo SamplingGenerating Random NumbersSimulating Random VariablesSimulating Random Vectors and ProcessesResamplingMarkov Chain Monte CarloMonte Carlo EstimationCrude Monte CarloBootstrap MethodVariance ReductionMonte Carlo for OptimizationSimulated AnnealingCross-Entropy MethodSplitting for OptimizationNoisy Optimization Exercises4. Unsupervised LearningIntroductionRisk and Loss in Unsupervised LearningExpectation–Maximization (EM) AlgorithmEmpirical Distribution and Density EstimationClustering via Mixture ModelsMixture ModelsEM Algorithm for Mixture ModelsClustering via Vector QuantizationK-MeansClustering via Continuous Multiextremal OptimizationHierarchical ClusteringPrincipal Component Analysis (PCA)Motivation: Principal Axes of an EllipsoidPCA and Singular Value Decomposition (SVD) Exercises5. RegressionIntroductionLinear RegressionAnalysis via Linear ModelsParameter EstimationModel Selection and PredictionCross-Validation and Predictive Residual Sum of SquaresIn-Sample Risk and Akaike Information CriterionCategorical FeaturesNested ModelsCoefficient of DeterminationInference for Normal Linear ModelsComparing Two Normal Linear ModelsConfidence and Prediction IntervalsNonlinear Regression ModelsLinear Models in PythonModelingAnalysisAnalysis of Variance (ANOVA)Confidence and Prediction IntervalsModel ValidationVariable SelectionGeneralized Linear Models Exercises6. Regularization and Kernel MethodsIntroductionRegularizationReproducing Kernel Hilbert SpacesConstruction of Reproducing KernelsReproducing Kernels via Feature MappingKernels from Characteristic FunctionsReproducing Kernels Using Orthonormal FeaturesKernels from Kernels 6.5 Representer TheoremSmoothing Cubic SplinesGaussian Process RegressionKernel PCA Exercises7. ClassificationIntroductionClassification MetricsClassification via Bayes’ RuleLinear and Quadratic Discriminant AnalysisLogistic Regression and Softmax ClassificationK-nearest Neighbors ClassificationSupport Vector MachineClassification with Scikit-Learn Exercises8. Decision Trees and Ensemble MethodsIntroductionTop-Down Construction of Decision TreesRegional Prediction FunctionsSplitting RulesTermination CriterionBasic ImplementationAdditional ConsiderationsBinary Versus Non-Binary TreesData PreprocessingAlternative Splitting RulesCategorical VariablesMissing ValuesControlling the Tree ShapeCost-Complexity PruningAdvantages and Limitations of Decision TreesBootstrap AggregationRandom ForestsBoosting ExercisesDeep LearningIntroductionFeed-Forward Neural NetworksBack-PropagationMethods for TrainingSteepest DescentLevenberg–Marquardt MethodLimited-Memory BFGS MethodAdaptive Gradient MethodsExamples in PythonSimple Polynomial RegressionImage Classification ExercisesA. Linear Algebra and Functional AnalysisVector Spaces, Bases, and MatricesInner ProductComplex Vectors and MatricesOrthogonal ProjectionsEigenvalues and EigenvectorsLeft- and Right-EigenvectorsMatrix Decompositions(P)LU DecompositionWoodbury IdentityCholesky DecompositionQR Decomposition and the Gram–Schmidt ProcedureSingular Value DecompositionSolving Structured Matrix EquationsFunctional AnalysisFourier TransformsDiscrete Fourier TransformFast Fourier TransformB. Multivariate Differentiation and OptimizationMultivariate DifferentiationTaylor ExpansionChain RuleOptimization TheoryConvexity and OptimizationLagrangian MethodDualityNumerical Root-Finding and MinimizationNewton-Like MethodsQuasi-Newton MethodsNormal Approximation MethodNonlinear Least SquaresConstrained Minimization via Penalty FunctionsC. Probability and StatisticsRandom Experiments and Probability SpacesRandom Variables and Probability DistributionsExpectationJoint DistributionsConditioning and IndependenceConditional ProbabilityIndependenceExpectation and CovarianceConditional Density and Conditional ExpectationFunctions of Random VariablesMultivariate Normal DistributionConvergence of Random VariablesLaw of Large Numbers and Central Limit TheoremMarkov ChainsStatisticsEstimationMethod of MomentsMaximum Likelihood MethodConfidence IntervalsHypothesis TestingD. Python PrimerGetting StartedPython ObjectsTypes and OperatorsFunctions and MethodsModulesFlow ControlIterationClassesFilesNumPyCreating and Shaping ArraysSlicingArray OperationsRandom NumbersMatplotlibCreating a Basic PlotPandasSeries and DataFrameManipulating Data FramesExtracting InformationPlottingScikit-learnPartitioning the DataStandardizationFitting and PredictionTesting the ModelSystem Calls, URL Access, and Speed-UpBibliographyIndexReviews“The first impression when handling and opening this book at a random page is superb. A big format (A4) and heavy weight, because the paper quality is high, along with a spectacular style and large font, much colour and many plots, and blocks of python code enhanced in colour boxes. This makes the book attractive and easy to study…The book is a very well-designed data science course, with mathematical rigor in mind. Key concepts are highlighted in red in the margins, often with links to other parts of the book…This book will be excellent for those that want to build a strong mathematical foundation for their knowledge on the main machine learning techniques, and at the same time get python recipes on how to perform the analyses for worked examples.“ – Victor Moreno, ISCB News, December 2020Authors BiographyDirk P. Kroese, PhD, is a Professor of Mathematics and Statistics at The University of Queensland. He has published over 120 articles and five books in a wide range of areas in mathematics, statistics, data science, machine learning, and Monte Carlo methods. He is a pioneer of the well-known Cross-Entropy method—an adaptive Monte Carlo technique, which is being used around the world to help solve difficult estimation and optimization problems in science, engineering, and finance.Zdravko Botev, PhD, is an Australian Mathematical Science Institute Lecturer in Data Science and Machine Learning with an appointment at the University of New South Wales in Sydney, Australia. He is the recipient of the 2018 Christopher Heyde Medal of the Australian Academy of Science for distinguished research in the Mathematical Sciences.Thomas Taimre, PhD, is a Senior Lecturer of Mathematics and Statistics at The University of Queensland. His research interests range from applied probability and Monte Carlo methods to applied physics and the remarkably universal self-mixing effect in lasers. He has published over 100 articles, holds a patent, and is the coauthor of Handbook of Monte Carlo Methods (Wiley).Radislav Vaisman, PhD, is a Lecturer of Mathematics and Statistics at The University of Queensland. His research interests lie at the intersection of applied probability, machine learning, and computer science. He has published over 20 articles and two books.
