ML Blog

Welcome to my research blog, structured like a library of books. Each book covers a major AI topic; every chapter is a self-contained post with explanations, worked examples, and references. Start with the Start Here overview of any book, then dive into whichever chapters interest you most.

โ˜… My papers Gold cards mark posts directly tied to my own research papers and companion explainers.
๐Ÿง 

Book 0, Basics

42 chapters on classical ML, clustering, dimensionality reduction and neural-network foundations

Start Here ยท Overview

Machine Learning Before Transformers: A Working Foundation

A detailed route through 42 ML chapters: learning and validation, classical prediction, trees and boosting, twelve clustering methods, PCA, dimensionality reduction and neural networks.

๐Ÿ“– 11 min read Seven sections, including 12 dedicated clustering methods and an expanded PCA chapter
Learning Foundations & Experimental Practice
Linear, Local & Probabilistic Prediction
๐Ÿ“ˆ

Linear Regression: Least Squares as Projection

Fitting a line by least squares is not an optimisation trick, it is the orthogonal projection of the o...

โฑ 14 min linear-regressionleast-squares
๐ŸŽฏ

Logistic Regression: Linear in the Log-Odds

Logistic regression is not a squashed linear regression, it is a straight line drawn in log-odds space...

โฑ 14 min logistic-regressionclassification
๐Ÿ“

Support Vector Machines: Margins and the Kernel Trick

Among all the hyperplanes that separate two classes, one sits furthest from both. Finding it turns out...

โฑ 8 min svmkernel-trick
๐Ÿ“

Ridge Regression: Shrinkage, Singular Directions and Stable Prediction

Derive the regularised least-squares solution, inspect shrinkage in the SVD basis, and connect scaling...

โฑ 5 min machine-learningregression
โœ‚๏ธ

Lasso Regression: Soft Thresholding, Sparse Solutions and Correlated Features

Derive the coordinate update from the L1 subgradient, calculate exact zeros, and explain when sparse s...

โฑ 5 min machine-learningregression
๐Ÿ‘ฅ

K-Nearest Neighbours: Local Prediction, Distance Weighting and Dimensionality

Compute classification and regression predictions, analyse the neighbourhood-size tradeoff, and unders...

โฑ 5 min machine-learningclassification
๐ŸŽฒ

Naive Bayes: Conditional Independence, Smoothing and Log-Space Prediction

Derive the class score, calculate a smoothed text example, and distinguish Gaussian, multinomial and B...

โฑ 5 min machine-learningclassification
Decision Trees & Ensembles
Clustering: One Method at a Time
๐Ÿ”

Clustering: What Each Algorithm Assumes a Cluster Is

k-means says a cluster is a ball around a centroid, DBSCAN says it is a connected dense region, and a ...

โฑ 23 min clusteringk-means
๐ŸŽฏ

K-means: Deriving Lloydโ€™s Algorithm, Choosing Centres and Diagnosing Failure

Derive the assignment and centroid updates, run a complete example, understand k-means++ seeding, and ...

โฑ 5 min machine-learningclustering
โšก

Mini-Batch K-means: Online Centroid Updates and the Cost of Approximation

Derive the running-mean update, distinguish mini-batch optimisation from concept-drift tracking, and m...

โฑ 5 min machine-learningclustering
๐Ÿ“

K-medoids: Representative Observations, Dissimilarities and PAM Swaps

Build a clustering around actual observations, calculate medoid costs, and understand the robustness a...

โฑ 5 min machine-learningclustering
๐ŸŒฟ

Agglomerative Clustering: Linkages, Wardโ€™s Criterion and Dendrograms

Compute competing linkage decisions, derive Wardโ€™s merge cost, and read a dendrogram without confusing...

โฑ 5 min machine-learningclustering
๐Ÿ”Ž

DBSCAN: Core Points, Density Reachability and Explicit Noise

Trace density-connected components by hand, distinguish core and border observations, and understand w...

โฑ 6 min machine-learningclustering
๐Ÿ“‰

OPTICS: Reachability Ordering and Clusters at Multiple Density Scales

Compute core and reachability distances, interpret valleys in the ordering, and distinguish the OPTICS...

โฑ 6 min machine-learningclustering
๐ŸŒฒ

HDBSCAN: Mutual Reachability, Condensed Trees and Stable Clusters

Build the density hierarchy from mutual reachability, explain cluster persistence and minimum-size pru...

โฑ 5 min machine-learningclustering
๐Ÿ””

Gaussian Mixture Models: Responsibilities, EM and Covariance Geometry

Derive expectationโ€“maximisation updates, calculate a soft assignment, and diagnose likelihood singular...

โฑ 5 min machine-learningclustering
๐Ÿ•ธ๏ธ

Spectral Clustering: Affinity Graphs, Laplacians and Relaxed Cuts

Derive the graph-smoothness objective, connect eigenvectors to disconnected components, and explain wh...

โฑ 5 min machine-learningclustering
๐Ÿ”๏ธ

Mean Shift: Kernel Density Modes, Bandwidth and Attraction Basins

Derive the weighted-mean movement behind mode seeking, follow a local update, and explain why bandwidt...

โฑ 5 min machine-learningclustering
๐ŸŒณ

BIRCH: Clustering Features, CF Trees and Streaming Compression

Derive the additive clustering-feature summary, calculate a subcluster radius, and understand what a C...

โฑ 5 min machine-learningclustering
๐ŸŒซ๏ธ

Fuzzy C-means: Membership Weights, Fuzziness and Alternating Updates

Derive the membership and centre updates, calculate a two-centre assignment, and distinguish fuzzy mem...

โฑ 5 min machine-learningclustering
๐Ÿงช

Evaluating Clustering: Silhouettes, Stability, External Labels and Useful Partitions

Calculate silhouette and pair agreement, diagnose misleading cluster-count criteria, and design evalua...

โฑ 5 min machine-learningclustering
PCA, Factorisation & Neighbourhood Embeddings
๐Ÿ“‰

PCA: Maximum Variance and Minimum Reconstruction Error

PCA can be derived by asking for the directions of greatest spread, or by asking for the subspace that...

โฑ 11 min pcadimensionality-reduction
๐ŸŒ€

Kernel PCA: Centred Gram Matrices, Nonlinear Coordinates and Pre-Images

Derive kernel centring and feature-space coordinates, project new observations correctly, and distingu...

โฑ 5 min machine-learningdimensionality-reduction
๐ŸŽ›๏ธ

Independent Component Analysis: Whitening, Non-Gaussianity and Source Recovery

Explain the linear mixing model, why decorrelation is insufficient, and how non-Gaussian contrasts ide...

โฑ 5 min machine-learningdimensionality-reduction
๐Ÿงฉ

Nonnegative Matrix Factorisation: Additive Parts, Loss Functions and Ambiguity

Derive nonnegative low-rank reconstruction, inspect multiplicative updates and scale ambiguity, and se...

โฑ 5 min machine-learningdimensionality-reduction
๐ŸŽฒ

Random Projections: Johnsonโ€“Lindenstrauss Geometry and Fast Compression

Derive expected squared-distance preservation, interpret finite-set concentration bounds, and compare ...

โฑ 5 min machine-learningdimensionality-reduction
๐Ÿ—บ๏ธ

t-SNE: Neighbour Probabilities, Perplexity and Reading the Map Correctly

Derive high- and low-dimensional affinities, explain the asymmetric KL objective, and identify which d...

โฑ 5 min machine-learningdimensionality-reduction
๐Ÿ”—

UMAP: Fuzzy Neighbourhood Graphs, Layout Optimisation and New-Point Transforms

Explain local distance normalisation, fuzzy graph combination and attractiveโ€“repulsive layout, then co...

โฑ 6 min machine-learningdimensionality-reduction
Neural Networks
Activation Functions
๐Ÿค–

Book I, Transformers

From the attention mechanism to GPT, BERT, ViT, and beyond

Start Here ยท Overview

Transformers: The Architecture That Changed AI

A self-contained guide to the Transformer, the engine behind GPT, BERT, and modern AI. Learn how attention replaces recurrence and why every major AI system uses it.

๐Ÿ“– 5 min read The complete picture in one post
๐Ÿงฉ Core Components
๐Ÿงฉ

The Transformer Block: Putting It All Together

A single Transformer block combines attention, residuals, layer norm, and an FFN into one reusable uni...

โฑ 6 min transformer-blockarchitecture
๐Ÿง 

Feed-Forward Networks: The Forgotten Half of Transformers

The FFN block holds two-thirds of a Transformer's parameters and does most of its factual recall. Yet ...

โฑ 7 min FFNMLP
โž•

Residual Connections: Why Transformers Can Be Deep

Without residual connections, training a 96-layer Transformer would be practically impossible. The ski...

โฑ 7 min residualskip-connections
๐Ÿ“Š

Layer Normalization in Transformers

Layer norm is not optional plumbing. It determines training stability, gradient flow, and whether deep...

โฑ 7 min layer-normbatch-norm
๐Ÿ›๏ธ

Encoder vs Decoder vs Encoder-Decoder Transformers

BERT, GPT, and T5 are all Transformers, but their architectures are fundamentally different. One compa...

โฑ 6 min BERTGPT
๐Ÿ”—

Cross-Attention: How Models Attend to Another Sequence

Cross-attention lets one sequence query information from a completely different sequence. It is the br...

โฑ 5 min attentioncross-attention
๐ŸŽญ

Attention Masks: Causal, Padding, and Bidirectional

The difference between GPT, BERT, and T5 is largely a masking decision. Learn how causal, padding, and...

โฑ 5 min attentionmasking
๐Ÿ”

Query, Key, Value: The Intuition Behind QKV

Q, K, and V are not arbitrary labels. They map precisely onto search queries, database labels, and ret...

โฑ 6 min attentionQKV
โš–๏ธ

Scaled Dot-Product Attention: Why the โˆšd Matters

Dividing by โˆšd_k is not just a trick, it prevents softmax from saturating and dying in high-dimensiona...

โฑ 4 min attentionscaling
๐Ÿ‘๏ธ

Multi-Head Attention: Many Eyes on the Data

One attention head sees one relationship. Multiple heads running in parallel let the model capture syn...

โฑ 4 min attentionmulti-head
๐Ÿ”

Self-Attention: Teaching Machines to Focus

Self-attention is the core of every Transformer. Learn how Query, Key, and Value vectors let every tok...

โฑ 5 min attentionmechanism
๐Ÿ—ฃ๏ธ

Let the Heads Talk: Beyond Diagonal Graph Attention

A bridge between two literatures that grew up apart, attention and sheaf neural networks. Viewed throu...

โฑ 27 min sheaf-neural-networksattention
๐Ÿ“ Positional Encodings
๐Ÿ”‘

GAPE: Remember to Forget, Gated Adaptive Positional Encoding

GAPE is a drop-in RoPE augmentation that adds content-aware attention logit biases: a query-gate suppr...

โฑ 9 min positional-encodingrope
๐Ÿ”ญ

LongRoPE: Extending Context to 2 Million Tokens

LongRoPE (Microsoft, 2024) pushes RoPE-based context to 2M tokens by searching for optimal per-dimensi...

โฑ 7 min RoPELongRoPE
๐Ÿงถ

YaRN: Yet Another RoPE Extensionn Method

YaRN combines NTK scaling for high-frequency dimensions with linear interpolation for low-frequency on...

โฑ 7 min RoPEYaRN
๐Ÿ“ก

NTK-Aware Scaling: Extending Context Without Fine-Tuning

NTK-Aware Scaling extends the context window of RoPE-based models by rescaling frequencies using Neura...

โฑ 6 min RoPENTK
๐Ÿ“

ALiBi: Attention with Linear Biases

ALiBi skips traditional positional embeddings entirely and just subtracts a distance penalty from atte...

โฑ 4 min positional-encodingalibi
๐Ÿ”„

RoPE: Rotary Position Embeddings

RoPE encodes position by rotating query and key vectors by an angle proportional to position. The clev...

โฑ 4 min positional-encodingrope
โ†”๏ธ

Relative Positional Encodings: It's All About Distance

Instead of asking 'where am I?', relative PEs ask 'how far are these two tokens apart?' Shaw et al. an...

โฑ 4 min positional-encodingrelative
๐ŸŽ“

Learned Positional Encodings: Data-Driven Position

Instead of a fixed formula, why not just train position embeddings from scratch, like word embeddings?...

โฑ 3 min positional-encodinglearned
ใ€ฐ๏ธ

Sinusoidal Positional Encodings: The Original Solution

The PE method from the 2017 'Attention Is All You Need' paper uses sine and cosine waves at different ...

โฑ 4 min positional-encodingsinusoidal
๐Ÿ“

Positional Encodings: Why Position Matters

Transformers see all tokens at once, which means without help they'd treat 'cat ate mouse' and 'mouse ...

โฑ 3 min positional-encodingoverview
๐ŸŒŠ

FoPE: Fourier Position Embedding for Length Generalization

FoPE rethinks long-context positional encoding from a frequency-domain perspective. Instead of only st...

โฑ 5 min FoPEpositional-encoding
๐Ÿชœ

Position Interpolation: Extending RoPE with Minimal Fine-Tuning

Position Interpolation rescales positions before applying RoPE so a model trained on short contexts ca...

โฑ 5 min RoPEposition-interpolation
๐Ÿงญ

XPos: Length-Extrapolatable Rotary Embeddings

XPos modifies RoPE with a multiplicative decay that keeps relative rotations while stabilising magnitu...

โฑ 4 min XPosRoPE
๐ŸŒ€

p-RoPE: What Makes Rotary Positional Encodings Useful?

This paper does two things at once: it explains what RoPE is really doing inside a trained LLM, and it...

โฑ 6 min p-RoPERoPE
๐Ÿ•ธ๏ธ

Book II, Graph Neural Networks

Graphs, spectral theory, and learning architectures for relational data

Start Here ยท Overview

Graph Neural Networks: Learning on Graphs

Graphs are everywhere, molecules, social networks, road maps, knowledge bases. Graph Neural Networks learn from this relational structure by propagating information between connected nodes. Here's the comple...

๐Ÿ“– 5 min read The complete picture in one post
๐Ÿ“Š Graph Fundamentals
๐Ÿ—๏ธ Architectures
๐Ÿ“จ

Message Passing: The Universal GNN Framework

Every GNN, GCN, GAT, GraphSAGE, GIN, is a special case of message passing. Learn the three-step loop t...

โฑ 5 min message-passingmpnn
๐Ÿ”ต

GCN: Graph Convolutional Networks

GCN (Kipf & Welling, 2016) is the 'hello world' of GNNs. It simplifies spectral graph convolution into...

โฑ 5 min gcnspectral
๐ŸŽฏ

GAT: Graph Attention Networks

GCN assigns the same (degree-based) weight to every neighbour. GAT learns which neighbours actually ma...

โฑ 5 min gatattention
๐ŸŒ

GraphSAGE: Inductive Learning on Large Graphs

GCN and GAT learn embeddings for fixed graphs, add a new node and you're stuck. GraphSAGE (Hamilton et...

โฑ 4 min graphsageinductive
โšก

GIN: Graph Isomorphism Network, The Most Expressive GNN

How powerful can a GNN be? Xu et al. (2019) answered with a theoretical bound, and GIN is the architec...

โฑ 5 min ginexpressiveness
๐Ÿ“

ChebNet: Spectral Graph Convolutions via Chebyshev Polynomials

ChebNet avoids the expensive full eigendecomposition by approximating spectral filters with Chebyshev ...

โฑ 6 min ChebNetspectral
โšก

SGC: Simple Graph Convolution

SGC removes all nonlinearities between GCN layers and collapses the entire propagation into a single p...

โฑ 7 min SGCsimple
๐Ÿ“Š

APPNP: Personalized PageRank Meets Graph Neural Networks

APPNP decouples feature transformation from propagation. A neural network transforms features first; t...

โฑ 7 min APPNPPageRank
๐ŸŒ

Graph Transformers: Bringing Attention to Graphs

Graph Transformers replace or augment local message passing with full pairwise attention, every node a...

โฑ 6 min graph-transformerattention
๐Ÿ†

Graphormer: Transformers with Structural Biases for Graphs

Graphormer encodes graph structure directly into Transformer attention via three biases: node centrali...

โฑ 9 min Graphormergraph-transformer
๐Ÿ“ฌ

MPNN: The General Message Passing Neural Network Framework

The MPNN framework (Gilmer et al., 2017) unifies GCN, GAT, GIN, GraphSAGE, and almost all spatial GNNs...

โฑ 7 min MPNNmessage-passing
๐Ÿ”ฌ Expressivity & Limitations
๐Ÿ“ Graph Positional & Structural Encodings
๐Ÿงบ Pooling & Graph-Level Learning
๐ŸŽจ Heterogeneous & Relational Graphs
๐ŸŒŠ Dynamic & Temporal Graphs
๐Ÿ”ฎ Geometric & Equivariant GNNs
๐Ÿš€ Applications
๐ŸŒฟ

Book III, Sheaf Neural Networks

From cellular sheaf theory to neural diffusion and attention

Start Here ยท Overview

Sheaf Neural Networks: A Complete Research Guide

Standard GNNs assume neighbouring nodes should agree. Sheaf Neural Networks replace that assumption with a learned linear map on every edge, which turns heterophily, oversmoothing, and directional structure ...

๐Ÿ“– 5 min read The complete picture in one post
๐Ÿงฑ Mathematical Foundations
๐Ÿ“„ Core Papers
๐Ÿง 

Neural Sheaf Diffusion: Heterophily and Oversmoothing Are the Same Problem

GNNs fail on heterophilic graphs and they oversmooth with depth. Bodnar et al. show these are one fail...

โฑ 13 min NSDneural-sheaf-diffusion
๐Ÿ‘๏ธ

Sheaf Attention Networks: GAT with Matrices Instead of Scalars

GAT weights a neighbour by a scalar. SheafAN keeps the scalar and adds a learned orthogonal transport ...

โฑ 9 min SheafANsheaf-attention
๐Ÿ“

PolyNSD: Polynomial Neural Sheaf Diffusion

PolyNSD replaces the NSD propagation operator with a degree-K Chebyshev polynomial in the normalised s...

โฑ 11 min sheaf-neural-networksspectral-gnn
๐Ÿ—๏ธ

DNSD: Making Sheaf Diffusion Work at Depth

Neural Sheaf Diffusion has a theoretical guarantee against representation collapse that does not survi...

โฑ 12 min sheaf-neural-networksdeep-gnns
๐Ÿ“

Conn-NSD: Computing the Sheaf Instead of Learning It

Neural Sheaf Diffusion learns the restriction maps by gradient descent. Conn-NSD computes them once, b...

โฑ 9 min sheaf-neural-networksconnection-laplacian
๐ŸŒŠ

Surfing on the Neural Sheaf: What Happens If You Use the Wave Equation

Every sheaf model so far discretises the heat equation, which dissipates energy. Suk et al. try the wa...

โฑ 9 min sheaf-neural-networkswave-equation
๐ŸŽง

Cooperative Sheaf Neural Networks: Listening Without Speaking

A sheaf gives every node a matrix-valued say in how its neighbours reach it, but not in whether they d...

โฑ 11 min sheaf-neural-networksdirected-graphs
๐Ÿงฒ

Equivariant Sheaf Neural Networks: Learning Geometric Transport on Graphs

Sheaf networks move features through matrix-valued maps but ignore the symmetries of physical space; e...

โฑ 32 min sheaf-neural-networksequivariance
๐Ÿ—ฃ๏ธ

Let the Heads Talk: Beyond Diagonal Graph Attention

A bridge between two literatures that grew up apart, attention and sheaf neural networks. Viewed throu...

โฑ 27 min sheaf-neural-networksattention
๐ŸŒŠ

ONDA: Oscillatory Neural Dynamics over Sheaves

ONDA combines learned sheaf transport with second-order wave dynamics: restriction maps shape how mess...

โฑ 14 min sheaf-neural-networksgraph-neural-networks
๐Ÿ”ฌ

Bochnerโ€“Weitzenbรถck for Sheaves: Transport, Curvature, and What Networks Learn

Our Bochnerโ€“Weitzenbรถck decomposition separates a sheaf Laplacian into anisotropic transport and a loc...

โฑ 19 min sheaf-neural-networksBochner-Weitzenbock
๐Ÿ”ญ Extensions
๐Ÿš€ Applications & Open Problems
๐ŸŽฎ

Book V, Reinforcement Learning

From MDPs and Bellman equations through deep RL, policy gradients, model-based methods, MARL, and RLHF

Start Here ยท Overview

Reinforcement Learning Basics: Agents, Rewards, and Returns

Start with the agentโ€“environment loop, delayed rewards, and the difference between a policy, a value function, and a model. Then follow a beginner route into the paper chapters.

๐Ÿ“– 4 min read Start with the basics, then explore the papers
๐Ÿงฑ Foundations
๐Ÿ“Š Value-Based Methods
๐ŸŽฏ Policy Gradient Methods
๐Ÿš€ Applications
๐Ÿค–

Book VI, Learning-Based Robotics

From kinematics and sensors through SLAM, imitation learning, sim-to-real, diffusion policy, and foundation models

Start Here ยท Overview

Robotics Basics: From Measurements to Motion

A beginner map of robot configuration, sensing, planning, control, and learning, with a clear distinction between deciding a task and commanding an actuator.

๐Ÿ“– 4 min read Start with the basics, then explore the papers
๐Ÿงฑ Foundations
๐Ÿ—บ๏ธ Planning & Navigation
๐ŸŽ“ Learning for Robots
๐Ÿ”ฎ Frontier & Open Problems
๐ŸŽจ

Generative AI & Style Transfer

Diffusion models, latent manipulation, style alignment, and controllable generation.

๐ŸŒซ๏ธ

Book VII, Diffusion Models

From denoising a fixed corruption to score-based SDEs, guidance, fast samplers and flow matching

Start Here ยท Overview

Diffusion Models: Learning to Undo Noise

Destroying an image is easy and needs no learning at all. Diffusion models exploit that asymmetry: they define a trivial forward corruption, then train a network to walk it backwards one small step at a time.

๐Ÿ“– 4 min read Why undoing noise is easier than building an image
๐Ÿงฑ Foundations
๐Ÿงฎ Score-Based & Deterministic Views
๐ŸŽ›๏ธ Conditioning & Guidance
โšก Fast Sampling & Efficiency
โžก๏ธ Flow Matching
๐Ÿš€ Applications
๐ŸงŠ

Book VIII, Geometric Deep Learning

Why convolutions, attention, message passing and sheaves are one blueprint applied to different symmetries

Start Here ยท Overview

Geometric Deep Learning: One Blueprint Behind Every Architecture

CNNs, GNNs, Transformers and sheaf models look like separate inventions. They are the same recipe applied to different domains: identify the symmetry of your data, then build layers that respect it. This boo...

๐Ÿ“– 6 min read The recipe behind every other book on this page
๐Ÿงฑ Symmetry & the Blueprint
๐ŸŒ The Five Domains
๐Ÿ”— Architectures Unified
๐ŸŽฐ Hardware & Scaling
๐Ÿ›ก๏ธ

Book IX, Cybersecurity

Machine learning for software security: program graphs, vulnerability intelligence, and securing code that AI wrote

๐Ÿงฉ

Book X, JEPA

Joint-embedding predictive architectures: images, video, world models, audio, language, and generative extensions

Start Here ยท Overview

JEPA: Learning What to Predict in Representation Space

A guide to joint-embedding predictive architectures: what their targets mean, why prediction can collapse, and how the idea extends from images to video, robotics, and language.

๐Ÿ“– 5 min read From latent prediction to representations that support action
๐Ÿงฉ Images & Representation Learning
๐Ÿงฉ Video & World Models
๐Ÿงฉ Audio & Language
๐Ÿงฉ Generative Extensions
๐Ÿ’ฌ

Book XI, Natural Language Processing

59 chapters on language foundations, syntax, semantics, applications and retrieval-augmented generation

Start Here ยท Overview

Natural Language Processing: From Text to Structure and Meaning

A complete route through 59 NLP chapters: corpora, morphology, word vectors, language models, structured prediction, syntax, semantics, applications, retrieval-augmented generation and evaluation.

๐Ÿ“– 14 min read Fifteen sections, including 23 chapters on RAG and related research
๐Ÿ’ฌ Text, Corpora & Morphology
๐Ÿ’ฌ Lexical & Distributional Representations
๐Ÿ’ฌ Language Models & Decoding
๐Ÿ’ฌ Classification, Sentiment & Topics
๐Ÿ’ฌ Tagging & Information Extraction
๐Ÿ’ฌ Syntax & Parsing Algorithms
๐Ÿ’ฌ Semantics, Reference & Inference
๐Ÿ’ฌ Retrieval, Translation & Language Applications
๐Ÿ’ฌ RAG: Foundations & Index Design
๐Ÿ’ฌ RAG: Retrieval, Reranking & Query Expansion
๐Ÿ’ฌ RAG: Retrieval-Aware Language Models
๐Ÿ’ฌ RAG: Adaptive Retrieval & Search Agents
๐Ÿ’ฌ RAG: Trees, Graphs & Corpus Synthesis
๐Ÿ’ฌ RAG: Evaluation, Citations & Long Context
๐Ÿ’ฌ Evaluation, Multilinguality & Experiments
๐Ÿ“

A1, Mathematical Basics

Linear algebra and calculus for machine learning, built for a fast recap before interviews

Start Here ยท Overview

What Mathematics an ML Engineer Actually Needs

Almost every mathematical question asked in an ML interview reduces to two things: what a matrix does to space, and how to differentiate a composition. This book covers those two things properly and is hones...

๐Ÿ“– 6 min read What you actually need, and what you can safely forget
๐Ÿงฎ Linear Algebra
๐Ÿ“ˆ Calculus & Differentiation
๐Ÿ“ Norms & Distances
๐ŸŽฏ Convexity & Optimisation
๐Ÿ“Š

A2, Statistics Basics

Estimation, inference and the interpretations people get wrong under pressure

Start Here ยท Overview

Statistics Basics: Reasoning Backwards From Data to Model

Probability runs forwards: pick a model, predict the data. Statistics runs backwards, and backwards is harder, many models could have produced what you saw. Everything else in this book is machinery for hand...

๐Ÿ“– 6 min read From data back to the model that produced it
๐Ÿ“‹ Descriptive Foundations
๐ŸŽฏ Estimation
๐Ÿ”ฌ Inference & Testing
๐ŸŽฒ

A3, Probability Basics

The language every likelihood, generative model and loss function is written in

Start Here ยท Overview

Probability for ML Interviews: The Eight Ideas Worth Re-deriving

Almost every loss function in machine learning is a negative log-likelihood in disguise, and almost every model output is a distribution. This book rebuilds the probability you need to read those objects flu...

๐Ÿ“– 5 min read Axioms through to entropy
๐Ÿงฉ Foundations & Bayes
๐Ÿ“‰ Random Variables & Distributions
๐Ÿ“ Limit Theorems
๐Ÿ’ก Information Theory
๐Ÿ”บ

A4, Geometry Basics

Metric spaces, manifolds and symmetry, the geometry behind embeddings and equivariant models

Start Here ยท Overview

Geometry for Machine Learning: Why Shape Keeps Coming Back

Every embedding you have ever trained lives in a metric space, every dataset you have ever fitted sits near a surface far thinner than its ambient dimension, and every architecture you trust encodes a symmet...

๐Ÿ“– 5 min read Why geometry keeps reappearing in machine learning
๐Ÿ“ Euclidean Foundations
๐ŸŒ Curvature & Manifolds
โ™ป๏ธ Symmetry & Groups
โš›๏ธ

A5, Physics Basics

Mechanics, statistical physics and stochastic processes, where diffusion models and energy-based models come from

Start Here ยท Overview

Physics for Machine Learning: Why the Same Equations Keep Coming Back

Diffusion models, energy-based models and Hamiltonian Monte Carlo were not inspired by physics, they are physics, rewritten with a neural network in place of an analytic potential. Knowing which physics save...

๐Ÿ“– 5 min read The physics that keeps surfacing in machine learning
๐Ÿช Classical Mechanics
๐Ÿ”ฅ Statistical Mechanics & Entropy
๐ŸŒซ๏ธ Stochastic Processes
โ™ป๏ธ Symmetry & Conservation
๐Ÿ–ฅ๏ธ

A6, Computer Science Basics

Complexity, data structures and algorithms, the fundamentals technical interviews actually test

Start Here ยท Overview

Computer Science Basics for ML Engineers: What the Interview Actually Tests

An ML interview that asks you to invert a binary tree is not testing your tree knowledge, it is testing whether you can state a cost, defend it, and write a loop whose boundaries are right the first time. He...

๐Ÿ“– 7 min read What you are actually tested on, and what you can skip
โฑ๏ธ Complexity Foundations
๐Ÿ—‚๏ธ Data Structures
โš™๏ธ Algorithms
๐Ÿงต Memory & Concurrency
๐Ÿ

A7, Python Programming Primer

A detailed, example-first guide to writing Python properly, syntax through to idiom and performance

Start Here ยท Overview

Python from the Ground Up: What Actually Happens When You Run a Script

Python is not interpreted line by line, and `python` is not the language. Knowing what CPython compiles your file into, and where it puts the packages you install, explains most of the confusion beginners hi...

๐Ÿ“– 7 min read From running your first script to writing Python that reads like Python
๐Ÿงฑ Language Foundations
๐Ÿ“ฆ Built-in Data Structures
๐Ÿ” Functions & Generators
๐Ÿ›๏ธ Classes & the Data Model
๐Ÿงฐ Modules, Errors & the Standard Library
๐Ÿš€ Idiomatic & Fast Python