Equivariance: What It Means and Why It Matters
Published:
Groups and Symmetry
Intuition First: Think of a compass. No matter which direction you hold it, it still points north — the reading is invariant to how you rotate your body. Now think of your shadow: if you rotate 90°, your shadow rotates 90° too — the shadow is equivariant to your rotation. These two everyday observations capture the entire mathematical framework of geometric deep learning.
A group \(G\) is a set of transformations \(\{g\}\) with a composition rule, an identity, and inverses. The groups relevant to 3D geometry differ in exactly two questions: are translations included, and are reflections?
| Group | Rotations | Reflections | Translations |
|---|---|---|---|
| \(\mathrm{SO}(3)\) | yes | no | no |
| \(\mathrm{O}(3)\) | yes | yes | no |
| \(\mathrm{SE}(3)\) | yes | no | yes |
| \(\mathrm{E}(3)\) | yes | yes | yes |
| \(\mathrm{E}(n)\) | yes | yes | yes (in \(n\) dimensions) |
\(\mathrm{O}(3) = \{Q \in \mathbb{R}^{3\times 3} : Q^{\!\top}Q = I\}\), and \(\mathrm{SO}(3)\) is its subgroup with \(\det Q = +1\) — the “S” is for special, meaning determinant one, meaning no reflections. Adding translations turns \(\mathrm{O}(3)\) into \(\mathrm{E}(3)\) and \(\mathrm{SO}(3)\) into \(\mathrm{SE}(3)\); the “E” is for Euclidean.
For molecular tasks the choice between \(\mathrm{SE}(3)\) and \(\mathrm{E}(3)\) is substantive: an \(\mathrm{E}(3)\)-invariant model gives mirror-image molecules identical predictions, which is right for energy and wrong for anything that depends on chirality.
Invariance vs Equivariance
Let \(\rho_{\text{in}}\) and \(\rho_{\text{out}}\) be the representations of \(G\) on the input and output spaces — the concrete matrices by which a group element acts on each.
\(G\)-invariant: the output does not move when the input is transformed.
Example: molecular potential energy. Rotating the molecule doesn’t change its energy.
\(G\)-equivariant: the output transforms too, by the group’s action on the output space.
Example: atomic forces, where \(\rho_{\text{in}}(R) = \rho_{\text{out}}(R) = R\), so rotating the molecule rotates the forces.
Note: invariance is the special case of equivariance in which \(\rho_{\text{out}}\) is the trivial representation, \(\rho_{\text{out}}(g) = I\) for every \(g\). Every invariant map is equivariant; the converse is false. Conflating the two is the most common error in this area, and the formulas above are the cheapest way to avoid it: if the right-hand side has a \(\rho_{\text{out}}(g)\) in it, the property is equivariance, not invariance.
Note also that \(\rho_{\text{in}}\) and \(\rho_{\text{out}}\) need not coincide even when both are non-trivial. A model mapping coordinates to a quadrupole moment is equivariant with \(\rho_{\text{in}}(R) = R\) but \(\rho_{\text{out}}(R) = D^{2}(R)\), a \(5 \times 5\) matrix.
Why Equivariance Is Better Than Augmentation
Data augmentation approach: train on random rotations of the molecule, hoping the model learns rotational invariance from data.
Problems:
- Requires many rotations per sample → expensive
- The model might learn approximate invariance, not exact invariance
- Generalisation to unseen orientations is not guaranteed
Equivariant approach: build the constraint into the architecture. The model is exactly equivariant by design — for any input orientation, the output transforms correctly. No augmentation needed.
Practical advantage: on molecular benchmarks, equivariant models generally reach a given accuracy from substantially fewer training samples than augmentation-based ones. The size of the gap depends on the task and the symmetry group, so treat it as a consistent direction rather than a fixed factor.
Worked Example: Invariant vs Equivariant in 2D
Setup: two atoms at positions \(x_1 = (1, 0)\) and \(x_2 = (0, 1)\). Apply a 90° counter-clockwise rotation \(R : (a, b) \mapsto (-b, a)\).
After rotation: \(x_1' = (0, 1)\), \(x_2' = (-1, 0)\).
Invariant quantity — distance:
- Before: \(\lVert x_1 - x_2 \rVert = \lVert (1, -1) \rVert = \sqrt{2}\)
- After: \(\lVert x_1' - x_2' \rVert = \lVert (1, 1) \rVert = \sqrt{2}\) ✓ same
Equivariant quantity — force vector (suppose \(F = (0.5, -0.5)\) before rotation):
- After rotation: \(R F = (0.5, 0.5)\) — the force has rotated by 90° too
- A model that outputs \((0.5, -0.5)\) for the original and \((0.5, 0.5)\) for the rotated version satisfies \(\Phi(Rx) = R\,\Phi(x)\) and is equivariant.
- A model that outputs \((0.5, -0.5)\) regardless of orientation satisfies \(\Phi(Rx) = \Phi(x)\). That is invariance, and for a force it is simply wrong — it would predict the same force direction for a molecule lying in any orientation.
This is the pair to keep in mind: the same input transformation, two different correct-looking equations, and only one of them is the right specification for a vector-valued target.
Representations: Scalars, Vectors, Tensors
The representation \(\rho_{\text{out}}\) determines how the output transforms:
Scalar (\(\ell = 0\), invariant): a single number. Energy, charge, mass. Unchanged by rotation: \(\rho(R) = 1\).
Vector (\(\ell = 1\), equivariant): a 3D vector. Forces, velocities, dipole moment. Rotates with the molecule: \(\rho(R) = R\).
Rank-2 tensor: a \(3 \times 3\) matrix, e.g. the stress tensor or the polarisability. It transforms as \(T \mapsto R T R^{\!\top}\), which on the flattened 9-vector is \(\rho(R) = R \otimes R\).
Irreducible representations (irreps) of \(\mathrm{SO}(3)\): indexed by a degree \(\ell = 0, 1, 2, \dots\), with \(\rho(R) = D^{\ell}(R)\) the \((2\ell+1) \times (2\ell+1)\) Wigner-D matrix. So \(\ell = 0\) is a scalar (1 component) and \(\ell = 1\) is a vector (3 components).
A rank-2 tensor is not the same thing as an \(\ell = 2\) feature. The 9-dimensional representation \(R \otimes R\) is reducible: it decomposes as \(\ell = 0 \oplus \ell = 1 \oplus \ell = 2\), corresponding to the trace (1 component), the antisymmetric part (3 components), and the traceless symmetric part (5 components). It is that last 5-dimensional piece that is the \(\ell = 2\) irrep. Higher \(\ell\) captures finer angular detail at increasing computational cost.
Types of Equivariant Models
Type 1: Distance-based invariance Features: only interatomic distances and angles. Output: scalar only. Architectures: SchNet, DimeNet. Limitation: cannot output vectors (forces require equivariant outputs), and because distances are \(\mathrm{E}(3)\)-invariant, cannot distinguish mirror images.
Type 2: Vector-based equivariance (\(\mathrm{E}(n)\) / \(\mathrm{SE}(3)\)) Features: positions as vectors, combined with scalar features. Output: scalars + vectors (\(\ell \le 1\)). Architectures: EGNN, PaiNN, NequIP.
Type 3: Tensor field networks (full irreps) Features: spherical harmonics up to degree \(L\). Output: features at any degree \(\ell \le L\). Architectures: TFN, SE(3)-Transformers, MACE. Limitation: expensive. The number of admissible \((\ell_{\text{in}}, \ell_f, \ell_{\text{out}})\) tensor-product paths grows roughly as \(O(L^3)\), and each path contracts irreps of dimension up to \(2L+1\), so a naive implementation is far worse than linear in \(L\). Practical libraries exploit the sparsity of the Clebsch–Gordan coefficients to cut this down considerably.
Building Equivariant Layers
A layer stays equivariant if it is built only from:
- Equivariant linear maps — acting on the irrep index, i.e. mixing channels within a degree, never across degrees arbitrarily
- Invariant scalars (distances, norms, inner products) used as coefficients
- Tensor products of irreps, contracted with Clebsch–Gordan coefficients
The key constraint: never feed raw coordinates into an arbitrary MLP alongside scalars. \(\mathrm{MLP}(Rx) \ne R\,\mathrm{MLP}(x)\) in general, and one such layer destroys the equivariance of the whole network — the property is only as strong as its weakest layer.
Summary
| Concept | Definition | Example |
|---|---|---|
| Invariant | \(\Phi(\rho(g)x) = \Phi(x)\) | Potential energy |
| Equivariant | \(\Phi(\rho_{\text{in}}(g)x) = \rho_{\text{out}}(g)\Phi(x)\) | Forces |
| Relationship | Invariance is equivariance with \(\rho_{\text{out}} = I\) | — |
| Augmentation | Learn symmetry from data | Expensive, approximate |
| Architectural equivariance | Baked-in symmetry | Exact, sample-efficient |
| Scalar (\(\ell = 0\)) | \(\rho(R) = 1\) | Energy, charge |
| Vector (\(\ell = 1\)) | \(\rho(R) = R\) | Force, velocity |
| Degree \(\ell\) | \(\rho(R) = D^{\ell}(R)\), size \(2\ell+1\) | Quadrupole (\(\ell = 2\)) |
Equivariance is the mathematical foundation of geometric deep learning. Every architecture in the next posts — EGNN, SE(3)-Transformers, TFN — is a concrete instantiation of these principles.
References
- Bronstein, M. M., Bruna, J., Cohen, T., & Veličković, P. (2021). Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv 2021 (comprehensive treatment of group symmetries, equivariance, and irreducible representations in deep learning).
- Cohen, T. S., & Welling, M. (2016). Group Equivariant Convolutional Networks. ICML 2016 (G-CNNs: first systematic framework for equivariant networks on discrete symmetry groups).
- Kondor, R., & Trivedi, S. (2018). On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups. ICML 2018 (theoretical foundation for equivariant neural networks over compact groups).