MATH 680: Computation Intensive Statistics

Winter 2018

References


Optional, no required textbook:

  • ESL: The Elements of Statistical Learning (2nd Ed) by T. Hastie, R. Tibshirani and J. Friedman

  • PR: Pattern Recognition by Christopher Bishop

  • CVX: Convex Optimization by Boyd and Vandenberghe

  • R: Advanced R by H. Wickham

Lecture Schedule


Topic 1 Introduction Slides Notes PR 1, R 1-3
Topic 2 Matrix Decomposition, OLS, PCA, PCR and Ridge Slides Notes PR 3.1, ESL 3.1-3.2 3.4.1, R 4-5
Topic 3 Convex Sets and Functions Slides Notes CVX 2-3
Topic 4 Optimization Basics Slides Notes CVX 4
Topic 5 Cross Validation Notes ESL 7
Topic 6 Gradient Descent Slides Note1 Note2 CVX 9.1-9.4
Topic 7 Regression Tree Slides Notes ESL 9.2
Topic 8 Gradient Boosting Slides Notes ESL 10
Topic 9 Subgradient Slides
Topic 10 Subgradient Method Slides Notes
Topic 11 Lasso and invariants Slides Notes ESL 3 18
Topic 12 Proximal Gradient Descent Slides FISTA
Topic 13 Duality Slides Notes CVX 5
Topic 14 KKT conditions Slides Notes CVX 5
Topic 15 Support Vector Machines Notes ESL 4.5 5.8 12 PR 6 7 Uniqueness Matrix
Topic 16 Duality Uses Slides
Topic 17 MM and EM Notes ESL 8.5 PR 9
Topic 18 ADMM Notes


Final Project

References


  1. Google PageRank Algorithm

  2. Polynomial Curve Fitting and Ridge Regression

  3. Cross Validation

  4. Gradient Boosting (original paper)

  5. Gradient Boosting (review paper)