ML Blog
Welcome to my research blog, structured like a library of books. Each book covers a major AI topic; every chapter is a short, self-contained post you can read in 3โ5 minutes. Start with the Start Here overview of any book, then dive into whichever chapters interest you most.
Machine Learning Before Transformers: A Working Foundation
Every later book on this site assumes you already know what a loss is, why gradient descent works, and what a convolution buys you. This book supplies that, and follows one thread through it: how much struct...
Bias, Variance, and Regularisation: Why Models Fail to Generalise
A model can fail for two opposite reasons: it is too rigid to represent the truth, or so flexible that...
Gradient Descent and Backpropagation: How a Model Learns
Training is one loop: measure the loss, ask backpropagation which way is downhill, take a small step. ...
Linear Regression: Least Squares as Projection
Fitting a line by least squares is not an optimisation trick, it is the orthogonal projection of the o...
Logistic Regression: Linear in the Log-Odds
Logistic regression is not a squashed linear regression, it is a straight line drawn in log-odds space...
Trees, Forests, and Boosting: Axis-Aligned Everything
A decision tree chops feature space into axis-aligned boxes and predicts one number per box, which exp...
Support Vector Machines: Margins and the Kernel Trick
Among all the hyperplanes that separate two classes, one sits furthest from both. Finding it turns out...
Clustering: What Each Algorithm Assumes a Cluster Is
k-means says a cluster is a ball around a centroid, DBSCAN says it is a connected dense region, and a ...
PCA: Maximum Variance and Minimum Reconstruction Error
PCA can be derived by asking for the directions of greatest spread, or by asking for the subspace that...
Convolutions and CNNs: Weight Sharing as a Prior
A dense layer from a 224-by-224 colour image to 1000 units holds 150.5 million weights; a 3-by-3, 64-f...
RNNs, LSTMs, and GRUs: Sequence Models Before Attention
A recurrent network shares weights across time exactly as a convolution shares them across space. The ...
Modern Activation Functions: GELU, SiLU, Mish, and Smooth Gating
Once ReLU became the default, researchers started asking a better question: can we keep the easy optim...
Activation Functions in Neural Networks: Why Non-Linearity Matters
Activation functions are the reason neural networks can model curved decision boundaries instead of co...
Output and Gated Activations: Softmax, Sparsemax, GLU, and SIREN
The last activation in your network is not a modelling preference, it is a contract with your loss fun...
Transformers: The Architecture That Changed AI
A self-contained guide to the Transformer, the engine behind GPT, BERT, and modern AI. Learn how attention replaces recurrence and why every major AI system uses it.
The Transformer Block: Putting It All Together
A single Transformer block combines attention, residuals, layer norm, and an FFN into one reusable uni...
Feed-Forward Networks: The Forgotten Half of Transformers
The FFN block holds two-thirds of a Transformer's parameters and does most of its factual recall. Yet ...
Residual Connections: Why Transformers Can Be Deep
Without residual connections, training a 96-layer Transformer would be practically impossible. The ski...
Layer Normalization in Transformers
Layer norm is not optional plumbing. It determines training stability, gradient flow, and whether deep...
Encoder vs Decoder vs Encoder-Decoder Transformers
BERT, GPT, and T5 are all Transformers, but their architectures are fundamentally different. One compa...
Cross-Attention: How Models Attend to Another Sequence
Cross-attention lets one sequence query information from a completely different sequence. It is the br...
Attention Masks: Causal, Padding, and Bidirectional
The difference between GPT, BERT, and T5 is largely a masking decision. Learn how causal, padding, and...
Query, Key, Value: The Intuition Behind QKV
Q, K, and V are not arbitrary labels. They map precisely onto search queries, database labels, and ret...
Scaled Dot-Product Attention: Why the โd Matters
Dividing by โd_k is not just a trick, it prevents softmax from saturating and dying in high-dimensiona...
Multi-Head Attention: Many Eyes on the Data
One attention head sees one relationship. Multiple heads running in parallel let the model capture syn...
Self-Attention: Teaching Machines to Focus
Self-attention is the core of every Transformer. Learn how Query, Key, and Value vectors let every tok...
Let the Heads Talk: Beyond Diagonal Graph Attention
A bridge between two literatures that grew up apart, attention and sheaf neural networks. Viewed throu...
GAPE: Remember to Forget, Gated Adaptive Positional Encoding
GAPE is a drop-in RoPE augmentation that adds content-aware attention logit biases: a query-gate suppr...
LongRoPE: Extending Context to 2 Million Tokens
LongRoPE (Microsoft, 2024) pushes RoPE-based context to 2M tokens by searching for optimal per-dimensi...
YaRN: Yet Another RoPE Extensionn Method
YaRN combines NTK scaling for high-frequency dimensions with linear interpolation for low-frequency on...
NTK-Aware Scaling: Extending Context Without Fine-Tuning
NTK-Aware Scaling extends the context window of RoPE-based models by rescaling frequencies using Neura...
ALiBi: Attention with Linear Biases
ALiBi skips traditional positional embeddings entirely and just subtracts a distance penalty from atte...
RoPE: Rotary Position Embeddings
RoPE encodes position by rotating query and key vectors by an angle proportional to position. The clev...
Relative Positional Encodings: It's All About Distance
Instead of asking 'where am I?', relative PEs ask 'how far are these two tokens apart?' Shaw et al. an...
Learned Positional Encodings: Data-Driven Position
Instead of a fixed formula, why not just train position embeddings from scratch, like word embeddings?...
Sinusoidal Positional Encodings: The Original Solution
The PE method from the 2017 'Attention Is All You Need' paper uses sine and cosine waves at different ...
Positional Encodings: Why Position Matters
Transformers see all tokens at once, which means without help they'd treat 'cat ate mouse' and 'mouse ...
FoPE: Fourier Position Embedding for Length Generalization
FoPE rethinks long-context positional encoding from a frequency-domain perspective. Instead of only st...
Position Interpolation: Extending RoPE with Minimal Fine-Tuning
Position Interpolation rescales positions before applying RoPE so a model trained on short contexts ca...
XPos: Length-Extrapolatable Rotary Embeddings
XPos modifies RoPE with a multiplicative decay that keeps relative rotations while stabilising magnitu...
p-RoPE: What Makes Rotary Positional Encodings Useful?
This paper does two things at once: it explains what RoPE is really doing inside a trained LLM, and it...
Graph Neural Networks: Learning on Graphs
Graphs are everywhere, molecules, social networks, road maps, knowledge bases. Graph Neural Networks learn from this relational structure by propagating information between connected nodes. Here's the comple...
The Graph Adjacency Matrix: A Graph in Matrix Form
Before understanding GNNs, you need to understand how graphs are represented mathematically. The adjac...
The Graph Laplacian: Spectral Graph Theory Explained Simply
The Graph Laplacian is L = D - A. Its eigenvectors reveal the graph's community structure; its eigenva...
What Is a Graph? Nodes, Edges, Features, and Labels
A graph is a set of nodes connected by edges, but the power of GNNs comes from the features attached t...
Directed, Undirected, Weighted, and Heterogeneous Graphs
Not all graphs are equal. Directed edges, edge weights, multiple node/edge types, each variant require...
Homophily vs Heterophily: When Neighbours Are Similar or Different
Most GNNs assume nearby nodes are similar, the homophily assumption. When this breaks (heterophilic gr...
Graph Tasks: Node, Edge, and Graph-Level Prediction
GNNs can predict at three levels: properties of individual nodes, existence or type of edges, or prope...
Graph Fourier Transform: The Spectral View of Graphs
The Graph Fourier Transform decomposes a signal on a graph into frequency components using the Laplaci...
Message Passing: The Universal GNN Framework
Every GNN, GCN, GAT, GraphSAGE, GIN, is a special case of message passing. Learn the three-step loop t...
GCN: Graph Convolutional Networks
GCN (Kipf & Welling, 2016) is the 'hello world' of GNNs. It simplifies spectral graph convolution into...
GAT: Graph Attention Networks
GCN assigns the same (degree-based) weight to every neighbour. GAT learns which neighbours actually ma...
GraphSAGE: Inductive Learning on Large Graphs
GCN and GAT learn embeddings for fixed graphs, add a new node and you're stuck. GraphSAGE (Hamilton et...
GIN: Graph Isomorphism Network, The Most Expressive GNN
How powerful can a GNN be? Xu et al. (2019) answered with a theoretical bound, and GIN is the architec...
ChebNet: Spectral Graph Convolutions via Chebyshev Polynomials
ChebNet avoids the expensive full eigendecomposition by approximating spectral filters with Chebyshev ...
SGC: Simple Graph Convolution
SGC removes all nonlinearities between GCN layers and collapses the entire propagation into a single p...
APPNP: Personalized PageRank Meets Graph Neural Networks
APPNP decouples feature transformation from propagation. A neural network transforms features first; t...
Graph Transformers: Bringing Attention to Graphs
Graph Transformers replace or augment local message passing with full pairwise attention, every node a...
Graphormer: Transformers with Structural Biases for Graphs
Graphormer encodes graph structure directly into Transformer attention via three biases: node centrali...
MPNN: The General Message Passing Neural Network Framework
The MPNN framework (Gilmer et al., 2017) unifies GCN, GAT, GIN, GraphSAGE, and almost all spatial GNNs...
The Weisfeiler-Lehman Test: How Powerful Are GNNs?
The 1-WL graph isomorphism test provides the exact upper bound on message-passing GNN expressivity. GI...
Oversmoothing: When All Node Embeddings Become the Same
Stack enough GNN layers and all node embeddings converge to the same vector, making the model useless....
Oversquashing: When Too Much Information Passes Through Bottlenecks
Oversquashing occurs when exponentially many node features must be compressed into a fixed-size embedd...
Over-smoothing vs Over-squashing: The Difference
Oversmoothing and oversquashing are both problems with deep GNNs, but they affect different nodes, hav...
Why Some Graphs Fool GNNs: The Structural Indistinguishability Problem
Certain graph structures are invisible to message-passing GNNs, not because of bad training, but becau...
Depth in GNNs: Why Deeper Is Not Always Better
In Transformers, depth = expressiveness. In GNNs, depth = both expressiveness AND over-smoothing. The ...
Why GNNs Need Positional Encodings
Message-passing GNNs are permutation-equivariant by design, they cannot assign unique positions to nod...
Laplacian Eigenvectors as Graph Positional Encodings
The k smallest eigenvectors of the graph Laplacian form a natural positional embedding space, the grap...
Random Walk Positional Encodings
Random walk positional encodings encode each node's structural context by computing the probability of...
Shortest-Path Encodings for Graph Transformers
Shortest-path distances between nodes can be encoded as attention biases or node features, directly in...
Structural vs Positional Encodings in Graphs
Positional encodings say where a node is in the graph. Structural encodings say what role it plays. Th...
Sign Ambiguity in Laplacian Eigenvectors
Laplacian eigenvectors are only defined up to sign: if u is an eigenvector, so is -u. This seemingly m...
Global Pooling in GNNs: Mean, Sum, and Max
To predict a property of an entire graph, node embeddings must be aggregated into a single vector. The...
DiffPool: Learning Hierarchical Graph Pooling
DiffPool learns to hierarchically cluster nodes into super-nodes across layers, like a convolutional p...
TopKPool and SAGPool: Sparse Graph Pooling
Instead of soft cluster assignment (DiffPool), TopKPool and SAGPool select a subset of the most import...
Set2Set and Attention Readout: Order-Invariant Graph Summaries
Mean and sum readout treat all nodes equally. Attention readout learns which nodes matter most for a g...
Graph Classification: From Node Embeddings to Graph Embeddings
Graph classification is the task of predicting a label for an entire graph. It requires composing mess...
Heterogeneous Graphs: When Nodes and Edges Have Types
Most real-world graphs are heterogeneous, they contain multiple node types (users, items, tags) and ed...
R-GCN: Relational Graph Convolutional Networks
R-GCN extends GCN to multi-relational graphs by learning a separate weight matrix for each relation ty...
HAN: Heterogeneous Graph Attention Networks
HAN combines meta-path decomposition with two levels of attention: node-level attention weights neighb...
Knowledge Graph Embeddings vs GNNs
Knowledge graph completion can be solved with shallow KG embeddings (TransE, DistMult, ComplEx) or wit...
Temporal Knowledge Graphs: Facts That Change Over Time
Most knowledge graphs treat facts as timeless, but facts change. Barack Obama was president from 2009 ...
Static vs Dynamic Graphs: When Structure Changes Over Time
Most GNN research assumes a fixed graph. Real graphs evolve: edges appear and disappear, node features...
Temporal Graph Networks: Learning from Events
TGN (Temporal Graph Network) is the leading framework for continuous-time dynamic graphs. It maintains...
Graph Neural ODEs: Continuous-Time Graph Dynamics
Neural ODEs replace discrete layer-by-layer computation with continuous dynamics governed by a differe...
Spatio-Temporal GNNs: Learning on Graphs Through Time
Spatio-temporal GNNs combine spatial message passing with temporal sequence modelling. They are the do...
Why Geometry Matters in Graph Neural Networks
Many real-world graphs are embedded in 3D space, molecules, proteins, point clouds, crystal structures...
Equivariance: What It Means and Why It Matters
Equivariance formalises the idea that a function should 'commute with symmetry transformations.' A rot...
EGNN: E(n)-Equivariant Graph Neural Networks
EGNN achieves E(n)-equivariance with a simple update rule: positions updated via weighted sums of rela...
SE(3)-Transformers: Attention with 3D Symmetry
SE(3)-Transformers extend self-attention to 3D point clouds and molecular graphs while maintaining SE(...
Tensor Field Networks and Geometric Deep Learning
Tensor Field Networks (TFN) were the first architecture to achieve SE(3) equivariance using spherical ...
Molecular GNNs: Learning on Atoms and Bonds
Molecules are graphs. Molecular GNNs predict chemical properties from structure. The best models use 3...
GNNs for Molecules: Drug Discovery and Material Design
Graph neural networks are transforming computational drug discovery. Molecules are natural graphs, and...
GNNs for Recommender Systems
Recommendation is naturally a graph problem: users and items are nodes, interactions are edges. GNNs o...
GNNs for Social Networks: Influence, Communities, and Misinformation
Social networks are large sparse graphs with rich node features (user profiles) and heterogeneous edge...
GNNs for Traffic Forecasting
Traffic prediction is a canonical spatio-temporal graph task: sensors on roads form a fixed graph, and...
GNNs for Knowledge Graphs: Reasoning and Completion
Knowledge graphs encode human knowledge as typed entity-relation triples. GNNs enable structure-aware ...
GNNs for Robotics: Planning, Manipulation, and Multi-Agent Systems
Robots interact with structured environments: objects have relationships, joints form kinematic chains...
GNNs for Computer Vision: Scene Graphs and Beyond
Computer vision tasks increasingly require relational reasoning, understanding how objects relate to e...
Sheaf Neural Networks: A Complete Research Guide
Standard GNNs assume neighbouring nodes should agree. Sheaf Neural Networks replace that assumption with a learned linear map on every edge, which turns heterophily, oversmoothing, and directional structure ...
Neural Sheaf Diffusion: Heterophily and Oversmoothing Are the Same Problem
GNNs fail on heterophilic graphs and they oversmooth with depth. Bodnar et al. show these are one fail...
Sheaf Attention Networks: GAT with Matrices Instead of Scalars
GAT weights a neighbour by a scalar. SheafAN keeps the scalar and adds a learned orthogonal transport ...
PolyNSD: Polynomial Neural Sheaf Diffusion
PolyNSD replaces the NSD propagation operator with a degree-K Chebyshev polynomial in the normalised s...
DNSD: Making Sheaf Diffusion Work at Depth
Neural Sheaf Diffusion has a theoretical guarantee against representation collapse that does not survi...
Conn-NSD: Computing the Sheaf Instead of Learning It
Neural Sheaf Diffusion learns the restriction maps by gradient descent. Conn-NSD computes them once, b...
Surfing on the Neural Sheaf: What Happens If You Use the Wave Equation
Every sheaf model so far discretises the heat equation, which dissipates energy. Suk et al. try the wa...
Cooperative Sheaf Neural Networks: Listening Without Speaking
A sheaf gives every node a matrix-valued say in how its neighbours reach it, but not in whether they d...
Equivariant Sheaf Neural Networks: Learning Geometric Transport on Graphs
Sheaf networks move features through matrix-valued maps but ignore the symmetries of physical space; e...
Let the Heads Talk: Beyond Diagonal Graph Attention
A bridge between two literatures that grew up apart, attention and sheaf neural networks. Viewed throu...
HetSheaf: Heterogeneous Graphs Meet Cellular Sheaves
HetSheaf encodes graph heterogeneity directly in the sheaf data structure, type-aware stalks and restr...
SheafPool: Basis-Invariant Graph Readout for Sheaf Neural Networks
SheafPool solves a key missing piece in sheaf GNNs: graph-level pooling. Instead of averaging stalk ve...
Sheaf-Based Positional Encodings: Letting Node Features Into the Spectrum
Laplacian eigenvector positional encodings tell a node where it sits in the graph, but the graph Lapla...
Joint Diffusion and Rotation Invariance: Sheaves That Learn to Lie
Every sheaf network so far predicts restriction maps with an MLP on concatenated features, a universal...
Bayesian Sheaf Neural Networks: Putting a Distribution on the Geometry
If a sheaf neural network learns its geometry from data, it can learn the wrong geometry and have no w...
Sheaf Hypergraph Networks: Apparent Consensus in Higher-Order Relations
A graph edge relates two things. A hyperedge relates any number of them, and hypergraph networks aggre...
BrainDyn: Sheaves Meet Neural ODEs for Brain Dynamics
Brain regions do not encode information in a shared feature space, which is exactly the assumption sca...
Sheaf4Rec: What a Recommender Gains from a Vector Space per Node
Collaborative filtering represents every user and item as one static vector. Sheaf4Rec replaces each w...
Topological Deep Learning: From Persistent Homology to Higher-Order Message Passing
This book has two halves. One computes topological summaries of data and feeds them to a model. The other makes the topology itself the domain the network runs on. This guide sets out both, and the pipeline ...
Topological Deep Learning Is Not Topological Data Analysis
TDA computes a topological descriptor and hands it to a model; TDL makes the topological object the do...
Message Passing on Simplicial Complexes
A simplex has four kinds of neighbour rather than one, and separating them lets a network see the diff...
Beyond Simplices: Cell and Combinatorial Complexes
A simplicial complex cannot hold a benzene ring as a single cell, filling the hexagon costs three edge...
Diffusion Models: Learning to Undo Noise
Destroying an image is easy and needs no learning at all. Diffusion models exploit that asymmetry: they define a trivial forward corruption, then train a network to walk it backwards one small step at a time.
The Forward Process: A Corruption Engineered to Be Jumped Into
The forward process looks like the trivial half of diffusion, but every term in it is load-bearing. Dr...
The Reverse Process: Why a Gaussian Is Enough, and What the Network Should Predict
Reversing a corruption is generally intractable. Diffusion escapes by taking steps small enough that t...
DDPM Training: From a Variational Bound to Four Lines of PyTorch
The DDPM objective starts as a variational bound with T+1 KL terms and ends as a plain mean-squared er...
Score Matching and the SDE View: DDPM as One Discretisation Among Many
Noise prediction and score estimation are the same network in different units. Taking the step size to...
DDIM: Same Marginals, Fewer Steps, and a Latent Space Worth Having
DDPM's objective never actually required the forward process to be Markov, only that its marginals be ...
Latent Diffusion: Denoise Where the Information Is
Most of the bits in a photograph encode texture no one can see. Latent diffusion throws them away firs...
Samplers and Schedulers: Diffusion Sampling Is Numerical Integration
Once you see the reverse process as an ODE, the thousand-step sampling loop stops being a fact of life...
Distillation and Consistency Models: Getting to Four Steps
Better ODE solvers bottom out around ten evaluations because the trajectory is genuinely curved. To go...
Flow Matching: Training a Velocity Field Without Ever Solving an ODE
Continuous normalising flows were elegant and nearly untrainable, every gradient step needed an ODE so...
Rectified Flow: Straightening Trajectories to Buy Few-Step Sampling
A perfectly straight generative trajectory is exactly solvable in a single Euler step. Every extra sam...
Diffusion vs Flow Matching: Two Names for One Family
Flow matching is often presented as the successor to diffusion. It is more accurate, and more useful, ...
Geometric Deep Learning: One Blueprint Behind Every Architecture
CNNs, GNNs, Transformers and sheaf models look like separate inventions. They are the same recipe applied to different domains: identify the symmetry of your data, then build layers that respect it. This boo...
Grids: Why Translation Equivariance Forces Convolution
Convolution is not a clever idea someone had about images. It is the only linear map that commutes wit...
Groups: Equivariance Beyond Translation
Translation is one group. Swap it for rotations, reflections, or the rigid motions of 3-D space and th...
Geodesics: Learning on Curved Domains
On a surface there is no global grid to slide a filter along, and no canonical direction to call 'up'....
Gauges: When There Is No Shared Frame
The previous chapter ended on an arbitrary choice that could not be eliminated. Gauge theory's answer ...
What Mathematics an ML Engineer Actually Needs
Almost every mathematical question asked in an ML interview reduces to two things: what a matrix does to space, and how to differentiate a composition. This book covers those two things properly and is hones...
Matrices as Linear Maps: Span, Rank, and the Subspaces They Create
A matrix is not a grid of numbers, it is a map. Once you read it that way, rank, column space, null sp...
Eigenvectors, the Spectral Theorem, and Why the SVD Always Exists
Eigenvectors are the directions a matrix does not rotate, when they exist. The SVD asks a weaker quest...
The Derivative Is a Linear Approximation, and the Gradient Is a Covector
Treating the derivative as a slope stops working the moment there is more than one input. Treating it ...
Jacobians, Hessians, and Why Newton's Method Loses at Scale
The Jacobian tells you how a map distorts volume, which is exactly the term normalising flows have to ...
Matrix Calculus Without Sign Errors: The Identities and the Layout Trap
Most matrix-calculus mistakes are not mistakes about calculus. They are mistakes about which conventio...
Statistics Basics: Reasoning Backwards From Data to Model
Probability runs forwards: pick a model, predict the data. Statistics runs backwards, and backwards is harder, many models could have produced what you saw. Everything else in this book is machinery for hand...
Estimators, Bias and Variance: Why Unbiased Is Not the Same as Good
An estimator is a random variable, so it has a mean and a spread. Squared error splits exactly into th...
Maximum Likelihood: The Default Recipe for Turning Data Into Parameters
Pick the parameter under which the data you actually observed would have been least surprising. That s...
Confidence Intervals: The Interval Is Random, the Parameter Is Not
A 95% confidence interval does not say the parameter is 95% likely to be inside it. It says the recipe...
Hypothesis Testing: What a p-Value Actually Measures
A p-value answers one narrow question: if nothing were going on, how often would data look at least th...
The Bootstrap: Uncertainty When the Algebra Runs Out
If you cannot resample from the population, resample from your sample instead. That one substitution g...
Bayesian vs Frequentist: Two Meanings of the Word Probability
One school says probability is a long-run frequency, so parameters cannot have probabilities. The othe...
Probability for ML Interviews: The Eight Ideas Worth Re-deriving
Almost every loss function in machine learning is a negative log-likelihood in disguise, and almost every model output is a distribution. This book rebuilds the probability you need to read those objects flu...
Sample Spaces and the Kolmogorov Axioms: What a Probability Actually Is
Probability is not defined as a long-run frequency or a degree of belief. It is defined as a function ...
Conditional Probability and Bayes: Why a 99% Accurate Test Means Almost Nothing
A test catches 99% of a disease and false-alarms only 5% of the time. You test positive. The chance yo...
Random Variables: A Density Is Not a Probability
A probability density can be 2, or 200, and nothing is wrong. Getting clear on what a PDF actually is ...
Expectation and Variance: Linearity Is Free, Additivity Is Not
Expectation adds up no matter how tangled the dependence. Variance does not, and the correction term, ...
The Ten Distributions You Need, and the Geometry of the Multivariate Gaussian
Ten families cover almost everything you will meet in a modelling interview. Nine of them fit in a tab...
Geometry for Machine Learning: Why Shape Keeps Coming Back
Every embedding you have ever trained lives in a metric space, every dataset you have ever fitted sits near a surface far thinner than its ambient dimension, and every architecture you trust encodes a symmet...
Euclidean Space: Inner Products, Projections and the Margin
One bilinear form generates the whole of flat geometry: lengths, angles, orthogonality, projections an...
Linear Maps as Geometry: Rotate, Scale, Rotate
A matrix is not a table of numbers, it is a deformation of space. The SVD says every deformation is th...
Curvature: Why a Flat Map of the Earth Must Lie
Curvature at a point is one over the radius of the circle that best hugs the curve there. Push that id...
Manifolds and Tangent Spaces: What 'The Data Lies on a Manifold' Actually Claims
A manifold is a space that looks flat close up and can be curved or knotted globally. That gap between...
Riemannian Geometry: Geodesics, Exponential Maps and Why Trees Prefer Hyperbolic Space
Put an inner product on every tangent space and let it vary smoothly: that single object determines le...
Physics for Machine Learning: Why the Same Equations Keep Coming Back
Diffusion models, energy-based models and Hamiltonian Monte Carlo were not inspired by physics, they are physics, rewritten with a neural network in place of an analytic potential. Knowing which physics save...
Lagrangian Mechanics: Why Nature Optimises a Functional
Newton says a particle moves because a force pushes it. Lagrange says it moves along the path that mak...
Hamiltonian Dynamics: Phase Space, Liouville, and Why HMC Works
Hamiltonian Monte Carlo is not a heuristic that happens to move well. It is exact because Hamiltonian ...
Statistical Mechanics: The Boltzmann Distribution and the Cost of Z
Counting microstates gives you the Boltzmann distribution, and the Boltzmann distribution gives you so...
Entropy and Free Energy: From the Second Law to the ELBO
Thermodynamic entropy and Shannon entropy differ by a constant with units. Once you accept that, the E...
Computer Science Basics for ML Engineers: What the Interview Actually Tests
An ML interview that asks you to invert a binary tree is not testing your tree knowledge, it is testing whether you can state a cost, defend it, and write a loop whose boundaries are right the first time. He...
Arrays, Dynamic Arrays and Hash Tables: Where O(1) Comes From and What It Costs
A linked list has O(1) insertion and loses to an array anyway. Appending is O(1) but a single append c...
Stacks, Queues and Heaps: Three Restrictions That Buy You Speed
A heap is a tree with no pointers, just an array and two index formulas, and building one from n items...
Trees, Traversals and BSTs: Why Balance Is the Only Thing Standing Between You and a Linked List
A binary search tree gives O(log n) search only if it is balanced, and inserting sorted keys, the most...
Graphs: Representations, BFS vs DFS, and the Assumption Dijkstra Makes Silently
Dijkstra is greedy: it finalises the closest unvisited vertex and never revisits it. That is valid onl...
Sorting and Searching: The n log n Lower Bound, and How to Beat It Legally
No comparison sort can beat n log n, that is a theorem with a two-line counting proof, not a limit of ...
Recursion and Dynamic Programming: Two Conditions, One Filled Table
Dynamic programming is not a trick, it is a diagnosis: if a problem has optimal substructure and overl...
Python from the Ground Up: What Actually Happens When You Run a Script
Python is not interpreted line by line, and `python` is not the language. Knowing what CPython compiles your file into, and where it puts the packages you install, explains most of the confusion beginners hi...
Names, Not Boxes: Python Syntax, Variables and the Built-in Types
A Python variable is not a container that holds a value, it is a label stuck onto an object that lives...
Operators and Control Flow: `is` Is Not `==`, and `for` Has an `else`
`==` asks whether two objects are equal; `is` asks whether they are the same object. CPython's caching...
Lists and Tuples: Slicing, Mutation, and the Cost of Every Method
A Python list is a growable array of pointers, and that one implementation fact predicts every complex...
Dicts and Sets: Hashing, Defaults, and What Makes a Key Legal
A dict trades memory for the ability to skip the search entirely. Everything that follows, why keys mu...
Strings and f-strings: Immutability, Formatting, and the Quadratic Loop
Strings cannot be modified, so every method that looks like it edits one actually builds another. That...
Functions: Arguments, Scope, Closures, and the Default That Remembers
A `def` is executed, not declared: it builds a function object once, evaluating the default arguments ...
Comprehensions and Generators: Building Sequences Without Building Them
A comprehension builds the whole result before you touch any of it. Change one pair of brackets to par...
Classes and Objects: self, Attributes, and When Inheritance Is the Wrong Tool
A Python class is a factory for namespaces, not a sealed blueprint. Once you see where an attribute ac...
Dunder Methods: How Python's Protocols Replace Interfaces
Python has almost no interfaces to implement and no operators to declare. Instead, every piece of synt...
Modules, Packages, and How Python Actually Finds Your Code
An import is not a textual include, it executes a file once, caches the result, and binds a name. Almo...
Errors and Exceptions: EAFP, the Hierarchy, and Reading a Traceback
Exceptions in Python are not exceptional. They are a normal control-flow mechanism that the language l...
Files and Context Managers: Why `with` Is Not Optional
A file object that falls out of scope without with does get closed eventually, but 'eventually' means ...
The Standard Library Modules Worth Knowing Before You `pip install`
Python ships with a counter, a queue, a memoiser, a CLI parser, a test framework's smaller cousin, and...