Machine Learning Course
Chapter 1: Foundations of Probability: Discrete Variables
| Subtopic | Detail |
|---|---|
| 1.1 Basics of Probability & Random Variables | Introduction to probability theory, sample spaces, events, and the definition of a random variable. |
| 1.2 Discrete Variables and Distribution | Definition of discrete random variables and the Probability Mass Function (PMF). |
| 1.3 Common Discrete Distributions | Detailed study of the point (degenerate), uniform, Bernoulli, binomial, and Poisson distributions. |
| 1.4 Cumulative Distribution Function (CDF) | Definition and properties of the Cumulative Distribution Function (CDF) for discrete variables. |
Chapter 2: Foundations of Probability: Continuous Variables and Features
| Subtopic | Detail |
|---|---|
| 2.1 Continuous Variables and Distribution | Definition of continuous random variables and the Probability Density Function (PDF). |
| 2.2 Common Continuous Distributions | Detailed study of the uniform, exponential, gamma, and normal (Gaussian) distributions. |
| 2.3 Features of Probability Distribution | Expectation (mean) of discrete and continuous random variables. |
| 2.4 Moments and Measures of Spread | Variance and standard deviation of random variables. Introduction to joint distributions, independent variables, covariance, and correlation. |
Chapter 3: Advanced Probability Concepts and Data Comparison
| Subtopic | Detail |
|---|---|
| 3.1 The Central Limit Theorem (CLT) | Statement, significance, and applications of the Central Limit Theorem. |
| 3.2 Joint and Conditional Probability | Detailed look at joint distributions, conditional probability, and Bayes' Theorem. |
| 3.3 Distances Between Distributions | Introduction to measuring similarity between distributions. |
| 3.4 Kullback-Leibler (KL) Divergence | The mathematical definition and practical interpretation of Kullback-Leibler divergence for comparing distributions. |
Chapter 4: Data Challenges and Introduction to Learning Types
| Subtopic | Detail |
|---|---|
| 4.1 Big Data, Big Challenges & Data Types | Introduction to big data characteristics (V's), practical challenges, and types of data. |
| 4.2 Types of Machine Learning | Classification of learning: Supervised, Unsupervised, Semi-supervised, and Reinforcement Learning. |
| 4.3 Supervised and Unsupervised Paradigms | Deep dive into supervised learning and unsupervised learning. |
| 4.4 Semi-supervised and Reinforcement Learning | Understanding semi-supervised learning and an introduction to the Reinforcement Learning framework. |
Chapter 5: Parametric, Non-parametric Models, and Data Issues
| Subtopic | Detail |
|---|---|
| 5.1 Parametric Models | Models with a fixed number of parameters and introduction to Linear and Logistic Regression. |
| 5.2 Non-parametric Models | Models whose parameters grow with the data, exemplified by the k-Nearest Neighbors (kNN) Classifier. |
| 5.3 Data Quality Challenges | Dealing with noise, irrelevance (redundant features), and heterogeneity in data. |
| 5.4 Imbalanced Data and Discretization | Strategies for handling imbalanced data and the process of data discretization. |
Chapter 6: Data Handling: Exploration and Preprocessing
| Subtopic | Detail |
|---|---|
| 6.1 Data Exploration and Introduction to Working with Data | Initial data analysis, understanding the data structure, and the overall ML workflow. |
| 6.2 Feature Extraction and Preprocessing | Techniques for feature extraction and the steps in data preprocessing. |
| 6.3 Visualizations for ML | Techniques for Machine Learning Visualizations to aid exploration and feature understanding. |
| 6.4 Applications of Data Preprocessing | Case studies demonstrating the impact of proper data preparation on model performance. |
Chapter 7: Data Handling: Descriptive Statistics and Estimation
| Subtopic | Detail |
|---|---|
| 7.1 Univariate Statistical Measures | Calculating mean, standard deviation, and various quantiles to summarize data. |
| 7.2 Data Visualization: Histogram | Construction, interpretation, and role of the Histogram in visualizing data distribution. |
| 7.3 Kernel Density Estimation (KDE) | Introduction to non-parametric density estimation using the KDE technique. |
| 7.4 Feature Imputation Techniques | Methods for dealing with missing values including basic and advanced feature imputation. |
Chapter 8: Introduction to Supervised Learning and Model Evaluation
| Subtopic | Detail |
|---|---|
| 8.1 Supervised Learning Fundamentals | Formal definition, problem setup, and goals of supervised learning. |
| 8.2 The Bias-Variance Tradeoff | Understanding the fundamental tradeoff between model bias and model variance. |
| 8.3 Loss Functions and Cost Minimization | Different types of loss functions and the concept of cost minimization. |
| 8.4 Model Evaluation: Testing and Cross-Validation | Importance of separate testing sets and techniques like k-fold cross-validation. |
Chapter 9: Linear Regression Models
| Subtopic | Detail |
|---|---|
| 9.1 Simple Linear Regression (Regression I) | Model representation, cost function, and the ordinary least squares solution. |
| 9.2 Multiple Linear Regression and Interpretation | Extending to multiple features, matrix notation, and interpreting model coefficients. |
| 9.3 Regularized Linear Regression (Regression II) | Introduction to the problem of overfitting and the role of regularization. |
| 9.4 Ridge and Lasso Regression | Detailed treatment of L2 (Ridge) and L1 (Lasso) regularization techniques. |
Chapter 10: Classification Models I: Fundamentals and Logistic Regression
| Subtopic | Detail |
|---|---|
| 10.1 Classification Problem Setup | Formal definition of the classification task and evaluation metrics. |
| 10.2 Logistic Regression Model | The model structure, the sigmoid function, and the derivation of the cost function. |
| 10.3 Parameter Learning for Logistic Regression | Applying Gradient Descent for optimizing the logistic regression parameters. |
| 10.4 Multinomial Classification | Extending binary classification to multiple classes using the Softmax function. |
Chapter 11: Classification Models II: Support Vector Machines and Decision Trees
| Subtopic | Detail |
|---|---|
| 11.1 Introduction to Decision Trees | Structure, key terminology, and the concept of recursive partitioning. |
| 11.2 Learning Decision Trees | Algorithms for tree construction and handling continuous features. |
| 11.3 Support Vector Machines (SVM) I: Linear SVM | The concept of the maximum margin hyperplane and linear SVM optimization. |
| 11.4 Support Vector Machines (SVM) II: Non-linear SVM and Kernels | Dealing with non-linearly separable data and the Kernel Trick. |
Chapter 12: Advanced Statistical ML and the Kernel Trick
| Subtopic | Detail |
|---|---|
| 12.1 Statistical Introduction to Machine Learning | Model assumptions, statistical efficiency, and relationship with hypothesis testing. |
| 12.2 Model Selection and Inference | Techniques for choosing the best model and making statistical inferences. |
| 12.3 The Kernel Trick for Regression | Utilizing the Kernel Trick to create infinitely flexible models. |
| 12.4 Kernel Methods and Basis Functions | Theoretical foundations of kernel methods and basis functions. |
Chapter 13: Bayesian Statistics and Gaussian Processes
| Subtopic | Detail |
|---|---|
| 13.1 Introduction to Bayesian Statistics | Bayes' Theorem, concepts of prior and posterior distributions. |
| 13.2 Bayesian Inference | Interactive prior→posterior updating, MAP vs MLE, and MCMC trace visualization. |
| 13.3 Bayesian Regression | Posterior predictive distribution with uncertainty bands and function samples. |
| 13.4 Gaussian Processes | Kernel exploration (RBF, Linear, Periodic, RQ), posterior samples & log marginal likelihood. |
Chapter 14: Unsupervised Learning: Clustering and Dimensionality Reduction
| Subtopic | Detail |
|---|---|
| 14.1 Clustering Fundamentals | Definition, applications, and challenges of clustering. |
| 14.2 K-Means Clustering | Step-by-step convergence, WCSS, silhouette, Davies-Bouldin, Calinski-Harabasz, elbow plot. |
| 14.3 Dimensionality Reduction I: PCA | Explained & cumulative variance, eigen directions, projection & optional animation. |
| 14.4 Dimensionality Reduction II: Advanced Techniques | PCA vs t-SNE vs UMAP comparison with perplexity, neighbor & min-dist controls. |
Chapter 15: Introduction to Neural Networks
| Subtopic | Detail |
|---|---|
| 15.1 The Neuron Model and Architectures | The Perceptron, activation functions, and basic feedforward architectures. |
| 15.2 Deep Learning Fundamentals | Why deep networks, representation learning, and the role of depth. |
| 15.3 Aspects of Neural Networks I: Hyperparameters | Key network hyperparameters and regularization techniques. |
| 15.4 Aspects of Neural Networks II: Initialization | Strategies for weight initialization and Batch Normalization. |
Chapter 16: Optimization of Neural Networks
| Subtopic | Detail |
|---|---|
| 16.1 Backpropagation Algorithm | The core mechanism for training: Backpropagation and gradient calculation. |
| 16.2 Optimization of Neural Networks I: Gradient Descent | Stochastic Gradient Descent and Mini-Batch variations. |
| 16.3 Optimization of Neural Networks II: Advanced Optimizers | Momentum, AdaGrad, RMSProp, and Adam optimizers. |
| 16.4 Optimization Challenges | Dealing with vanishing/exploding gradients and saddle points. |
Chapter 17: Representation Learning and Autoencoders
| Subtopic | Detail |
|---|---|
| 17.1 Introduction to Representation Learning | Formal definition and the goals of learning useful features. |
| 17.2 Autoencoders I: Structure and Loss | The Autoencoder architecture and unsupervised dimensionality reduction. |
| 17.3 Autoencoders II: Variants | Denoising Autoencoders and Variational Autoencoders (VAEs). |
| 17.4 Applications of Representation Learning | Using learned representations for transfer learning and pre-training. |
Chapter 18: Computer Vision: CNNs and Image Analysis
| Subtopic | Detail |
|---|---|
| 18.1 Computer Vision Basics | Introduction to digital images and classical CV techniques. |
| 18.2 Convolutional Neural Networks (CNNs) I | The Convolutional Layer, feature maps, and weight sharing. |
| 18.3 Convolutional Neural Networks (CNNs) II | Pooling layers and the complete CNN architecture. |
| 18.4 Image Detection and Segmentation | Introduction to object detection and image segmentation. |
Chapter 19: Advanced Deep Learning Architectures
| Subtopic | Detail |
|---|---|
| 19.1 Graph Neural Networks (GNNs) I: Foundations | Introduction to graph data and Graph Neural Networks. |
| 19.2 Graph Neural Networks (GNNs) II: Advanced Models | Advanced GNN architectures for node and graph classification. |
| 19.3 Self-Supervised Learning (SSL) | Introduction to Self-Supervised Learning concepts. |
| 19.4 SSL Techniques and Applications | Common SSL pretext tasks and their applications. |
Chapter 20: Transformers and Large Language Models (LLMs)
| Subtopic | Detail |
|---|---|
| 20.1 Transformers I: Architecture and Attention | The Transformer model, Self-Attention and Positional Encoding. |
| 20.2 Transformers II: Encoders and Decoders | The full Transformer block structure and stack roles. |
| 20.3 Large Language Models (LLMs) I: Training and Scale | Large Language Models, pre-training, and scaling effects. |
| 20.4 Large Language Models (LLMs) II: Applications | Fine-tuning techniques and key applications of LLMs. |