Classical Machine Learning
Optimisation, regression, regularisation and model selection — the statistical toolkit that still wins most tabular problems.
- Lessons
- 8
- Study time
- 4.4 h
- Access
- First lesson free
Syllabus
- 2.1
Gradient descent, saddle points and optimizer dynamics
Race SGD, Momentum and Adam across a convex bowl, a saddle and the Rosenbrock valley, then write your own Adam and watch it roll.
40 minFreeLive 3D lab and Python challengeintermediate- Derive the gradient descent update and the role of the learning rate
- Explain how momentum accumulates velocity across a ravine
- Implement Adam's bias-corrected first and second moment estimates
- 2.2
Linear and logistic regression from first principles
Closed-form least squares as a projection, then the sigmoid, cross-entropy and the convex surface it creates.
30 minProLive 3D lab and Python challengeintermediate- Solve least squares with the normal equations and as a projection
- Derive the logistic regression gradient from cross-entropy
- Interpret coefficients, odds ratios and decision boundaries
- 2.3
Bias, variance and regularisation
Why more capacity can hurt, how ridge and lasso trade bias for variance, and how to choose λ with cross-validation.
28 minProLive 3D lab and Python challengeintermediate- Decompose expected error into bias, variance and noise
- Compare ridge and lasso geometrically
- Select hyperparameters with k-fold cross-validation
- 2.4
Decision trees, random forests and gradient boosting
Axis-aligned partitions of feature space, bagging to cut variance, and boosting as gradient descent in function space.
36 minProLive 3D lab and Python challengeintermediate- Grow a tree by greedy impurity reduction
- Explain why bagging decorrelates and stabilises predictions
- Read gradient boosting as functional gradient descent
- 2.5
Support vector machines and the kernel trick
Maximise the margin, then lift the data into a space where a plane can separate it, without ever computing that space.
34 minProLive 3D lab and Python challengeintermediate- Derive the maximum-margin objective and identify support vectors
- Trade margin width against violations with the C parameter
- Use polynomial and RBF kernels as implicit feature maps
- 2.6
k-means, Gaussian mixtures and EM
Unsupervised structure: Lloyd's algorithm, k-means++ seeding, and soft clustering with expectation-maximisation.
34 minProLive 3D lab and Python challengeintermediate- Run k-means and explain why it converges to a local optimum
- Fit a Gaussian mixture with expectation-maximisation
- Choose the number of clusters with inertia, silhouette and BIC
- 2.7
PCA and dimensionality reduction
Find the directions of greatest variance, project onto them, and see where a linear projection fails on a curled-up manifold.
32 minProLive 3D lab and Python challengeintermediate- Derive principal components as eigenvectors of the covariance matrix
- Compute PCA through the SVD and choose k from explained variance
- Recognise when nonlinear methods such as kernel PCA, t-SNE or UMAP are needed
- 2.8
Precision, recall, ROC and calibration
Slide a decision threshold through two score distributions and watch the confusion matrix, ROC and precision-recall curves respond.
32 minFree previewLive 3D lab and Python challengeintermediate- Compute precision, recall, F1 and specificity from a confusion matrix
- Read ROC and precision-recall curves, and explain why AUC is a ranking probability
- Pick a threshold from error costs and handle class imbalance