4.2 Types of Machine Learning
Machine Learning Taxonomy for LLM Development
Understanding different learning paradigms is crucial for developing effective Large Language Models. Each approach addresses specific challenges in natural language understanding and generation.
Supervised Learning: Learning from Examples
The foundation of most LLM training, where models learn from input-output pairs.
Mathematical Framework
Given training data D = {(x₁, y₁), (x₂, y₂), ..., (xₙ, yₙ)}, find function f: X → Y that minimizes:
Where ℓ is the loss function and Ω(f) is a regularization term.
LLM Applications
Next Token Prediction
Input: "The capital of France is"
Target: "Paris"
Objective: P(w_{t+1} | w₁, w₂, ..., wₜ)
Text Classification
Input: "This movie was amazing!"
Target: "Positive"
Objective: P(class | text)
Question Answering
Input: Context + Question
Target: Answer span
Objective: P(answer | context, question)
Interactive: Next Token Prediction
Top Predictions:
Unsupervised Learning: Finding Hidden Patterns
Learning from data without explicit targets, crucial for understanding language structure and representation learning.
Mathematical Framework
Given data X = {x₁, x₂, ..., xₙ}, discover hidden structure by optimizing:
Where θ represents model parameters and p(x|θ) is the data likelihood.
Key Techniques for LLMs
Clustering for Topic Discovery
Group similar texts to discover latent topics and themes in training corpora.
Algorithm: k-means with cosine similarity
Word Embeddings
Learn dense vector representations that capture semantic relationships.
Word2Vec Objective:
Autoencoder Language Models
Learn compact representations by reconstructing input text.
Objective: Minimize reconstruction loss
Interactive: t-SNE Visualization
Explore how high-dimensional word embeddings cluster in 2D space:
Reinforcement Learning: Learning from Feedback
Critical for aligning LLMs with human preferences and optimizing for complex objectives.
Mathematical Framework
Agent learns policy π(a|s) to maximize expected cumulative reward:
Where γ is the discount factor and R(s,a) is the reward function.
RLHF: Reinforcement Learning from Human Feedback
{{step.title}}
{{step.description}}
Interactive: Policy Learning Simulation
Self-Supervised Learning: The LLM Revolution
The dominant paradigm for modern LLMs, learning rich representations from raw text without explicit labels.
Mathematical Framework
Create supervision signal from the data itself through pretext tasks:
Where context(xᵢ) represents the surrounding tokens or masked portions.
Pretext Tasks for Language Models
{{task.name}}
{{task.description}}
{{task.input}}
{{task.target}}
Interactive: Masked Language Modeling
Masked Text:
Prediction Task:
Model must predict the [MASK] tokens based on context.
Comparative Analysis
| Learning Type | Data Requirement | Scalability | Performance | LLM Usage |
|---|---|---|---|---|
| {{comparison.type}} | {{comparison.data}} | {{comparison.scalability}} | {{comparison.performance}} | {{comparison.usage}} |
Modern LLM Training Pipeline
How different learning types combine in state-of-the-art language models:
{{stage.name}}
{{stage.type}}{{stage.description}}