__Discussion__**Wednesdays 3-4pm**in Etcheverry 3113- Email: kevintli@
- Anonymous feedback form
- Welcome survey - please fill this out if you plan on attending my discussions and/or if you'd like to receive occasional recap emails!

**Section Materials****Discussion 12: Principal Components Analysis (PCA)**Rayleigh quotients and their connection to the spectral norm and related optimization problems. Derivations of PCA through various methods: Gaussian MLE, maximizing variance, and minimizing projection error. Relationship between the SVD and PCA.

> Slides

**Discussion 11: Neural Networks**Neural network basics: feature/representation learning, universal function approximation, motivations for backprop, and how to derive gradients for functions involving matrices and batch dimensions.

>

**Slides****Discussion 10: Kernel Methods**Kernel methods and their motivation as both enabling efficient high-dimensional featurization, and allowing custom notions of similarity between data points. Conditions for the validity of a kernel function.

>

**Slides****Discussion 9: Decision Trees & Random Forests**Decision tree foundations: entropy, information gain, and strictly concave cost functions. Motivation behind random forests.

>

**Slides****Discussion 7: Midterm Review**Miscellaneous practice problems: logistic regression, squared vs. logistic vs. hinge loss functions, LDA/QDA, gradient descent and convexity

**Discussion 6: Least Squares & Least Norm**Least-squares linear regression and motivation for the min-norm solution in the case of infinitely many solutions. SVD, the Moore-Penrose Pseudoinverse, and its application to the min-norm least squares problem.

>

**Slides****Discussion 5: Anisotropic Gaussians, Transformations, Quadratic Forms**Overview of anisotropic Gaussians, including properties of the covariance matrix and the elliptical isocontours of the quadratic form. Change of basis as a way to understand various data transformations (sphering, whitening, etc.).

>

**Slides****Discussion 4: Generative Models, GDA, MLE**Review of Bayes Decision Theory and MLE, and their applications to generative modeling. Gaussian Discriminant Analysis (QDA/LDA) as a special case of generative models.

>

**Slides****Discussion 3: Soft-Margin SVMs, Decision Theory**Soft-margin SVMs, hinge loss, and interesting variants of SVMs for outlier detection. Deriving posterior class probabilities using Bayes' Rule.

>

**Slides****Discussion 2: Math Prep**Review of math concepts that are useful in machine learning: linear algebra, probability, and vector calculus (especially taking derivatives of matrix/vector functions).

>

**Slides****Discussion 1: Intro & SVMs**[recording]Review of vectors, projection, hyperplanes, and the distance formula. Intro to hard-margin SVMs, including motivation and formulation of the optimization problem.

*Additional resources***Understanding the SVM formulation**

Cookie Use

We use cookies to ensure a smooth browsing experience. By continuing we assume you accept the use of cookies.