8.1 Supervised Learning Fundamentals

Introduction

Supervised learning learns from labeled examples. Given input-output pairs \((x, y)\), the goal is to find a function \(f\) such that \(f(x) \approx y\) for new, unseen inputs. This is central to fine-tuning LLMs on specific tasks like text classification or question answering.

1. Core Concepts

Labeled Data

Training data consists of pairs: \(\mathcal{D} = \{(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)\}\)

• \(x_i\): Input features (e.g., text embeddings, token sequences)
• \(y_i\): Target labels (e.g., sentiment, next token, class)
• \(n\): Number of training examples

Learning Objective

Find function \(f\) that minimizes prediction error: $$\min_{f} \frac{1}{n}\sum_{i=1}^{n} L(y_i, f(x_i))$$ where \(L\) is a loss function measuring error between prediction \(f(x_i)\) and true label \(y_i\).

2. Classification vs Regression

Classification

Predict discrete labels from finite set: $$y \in \{1, 2, \ldots, K\}$$ Examples: Sentiment analysis (positive/negative), spam detection, language identification.

Regression

Predict continuous values: $$y \in \mathbb{R}$$ Examples: Predicting text quality scores, estimating reading time, forecasting engagement.

3. Training, Validation, and Test Sets

Data Split Strategy

Divide data into three disjoint sets:

• Training Set (60-80%): Used to fit model parameters
• Validation Set (10-20%): Tune hyperparameters, early stopping
• Test Set (10-20%): Final performance evaluation (never touch during development)

Why Split?

Prevents overfitting - model memorizing training data instead of learning generalizable patterns: $$\text{Test Error} = \text{Bias}^2 + \text{Variance} + \text{Noise}$$

4. Overfitting and Underfitting

Overfitting (High Variance)

Model fits training data too well, captures noise: $$\text{Training Error} \ll \text{Validation Error}$$ Signs: High accuracy on train, low on validation. Model too complex.

Underfitting (High Bias)

Model too simple, cannot capture underlying patterns: $$\text{Training Error} \approx \text{Validation Error} \gg 0$$ Signs: Poor performance on both sets. Model too simple.

Good Fit

Balanced complexity: $$\text{Training Error} \approx \text{Validation Error} \approx \text{Irreducible Error}$$

5. Learning Curves

Plot training and validation error vs training set size or epochs:

• Training curve decreases: Model learning patterns
• Validation curve converges: Generalization improving
• Gap between curves: Indicates overfitting

6. Cross-Validation Preview

Instead of single train/val split, use k-fold cross-validation:

1. Split data into \(k\) equal folds
2. Train on \(k-1\) folds, validate on remaining fold
3. Repeat \(k\) times, each fold used once for validation
4. Average performance across all folds

$$\text{CV Error} = \frac{1}{k}\sum_{i=1}^{k} \text{Error}_i$$ More robust estimate of generalization error (covered in detail in Model Evaluation).

7. Supervised Learning for LLMs

Fine-tuning

Start with pre-trained model, adapt to specific task with labeled data: $$\theta^* = \arg\min_{\theta} \sum_{i=1}^{n} L(y_i, f_{\theta}(x_i))$$ where \(\theta\) are model parameters initialized from pre-training.

Common LLM Supervised Tasks

• Text Classification: Sentiment, topic, intent detection
• Named Entity Recognition: Extract entities (people, places, organizations)
• Question Answering: Given context and question, predict answer span
• Sequence-to-Sequence: Translation, summarization, paraphrasing

Key Takeaways

• Supervised learning requires labeled data (input-output pairs)
• Split data into train/validation/test to evaluate generalization
• Overfitting occurs when model is too complex; underfitting when too simple
• Learning curves help diagnose model performance
• Cross-validation provides robust performance estimates
• LLM fine-tuning adapts pre-trained models to specific tasks