Alessio Borgi

Alessio Borgi

PhD in Graph Neural Networks and Generative AI 👨🏻‍💻🤖

Topological Autoencoders: Preserving Shape in Latent Space

2 minute read

Published:

TL;DR: A Topological Autoencoder (TopoAE) minimises the sum of a reconstruction loss and a topological similarity loss: λ·d(dgm(X), dgm(Z)), where X is the input point cloud and Z = encoder(X) is the latent point cloud. This forces the encoder to preserve homological features (number of clusters, loops) across the bottleneck. Unlike VAEs (which regularise towards Gaussian), TopoAEs regularise towards the actual topology of the data.

Standard Autoencoders and Topological Distortion

A standard autoencoder minimises \(\|x - \hat{x}\|^2\) (reconstruction loss). This encourages point-wise fidelity but makes no guarantees about the global structure of the latent space.

Problem: Methods like t-SNE and UMAP empirically preserve local structure (nearby points stay nearby) but can distort global topology — a single connected manifold can be fragmented into disconnected clusters, or a loop can be contracted to a point.

The TopoAE Objective

Topological Autoencoder (Moor et al. 2020) defines:

$$\mathcal{L}_{TopoAE} = \underbrace{\|X - \hat{X}\|^2}_{\text{reconstruction}} + \lambda \underbrace{d_{T}(\mathrm{dgm}(X), \mathrm{dgm}(Z))}_{\text{topological}}$$

where:

  • \(\mathrm{dgm}(X)\) = persistence diagram of the Rips filtration on input points \(X\).
  • \(\mathrm{dgm}(Z)\) = persistence diagram of the Rips filtration on latent codes \(Z = e_\theta(X)\).
  • \(d_T\) is a differentiable approximation to the bottleneck/Wasserstein distance.

Computing the Topological Loss

The Wasserstein distance between diagrams can be computed via linear assignment (the Hungarian algorithm). For batches, the key insight is:

  1. Each diagram point \((b_k, d_k)\) in \(\mathrm{dgm}(X)\) corresponds to a specific edge or simplex in the filtration.
  2. Each diagram point in \(\mathrm{dgm}(Z)\) similarly corresponds to latent-space distances.
  3. Matching diagram points gives a correspondence between input-space features and latent-space features.
  4. The loss measures how much the birth/death times shift.

Gradients flow back through: \(\mathrm{loss} \to d_k^{(Z)} \to d(z_i, z_j) \to z_i = e_\theta(x_i) \to \theta\)

What Topology Is Preserved

For \(H_0\) (connected components): the encoder is forced to map disconnected components in \(X\) to disconnected regions in \(Z\), and connected data to connected latent space.

For \(H_1\) (loops): if the input data lies near a circle, the latent space is forced to also have a circular topology, not a line.

Comparison with Other Methods

MethodPreserves Local StructurePreserves Global Topology
PCAPartiallyNo
t-SNEYesNo
UMAPYesPartially
VAENo guaranteeNo
TopoAEYes (via reconstruction)Yes (via topology loss)
Key Insight: The failure mode of standard autoencoders is that the latent space "folds" the data manifold — a torus can become a cylinder or even a disc in the latent space, destroying H₁ features. TopoAE's topological loss detects this folding (as new short-lived H₁ features or death of long-lived features) and penalises it, maintaining the "shape" of the data manifold throughout training.

References

  • M. Moor, M. Horn, B. Rieck, K. Borgwardt, “Topological Autoencoders,” ICML 2020. arXiv:1906.00722.
  • L. McInnes, J. Healy, J. Melville, “UMAP: Uniform Manifold Approximation and Projection,” arXiv:1802.03426.

You May Also Enjoy

GAPE: Remember to Forget — Gated Adaptive Positional Encoding

4 minute read

Published:

GAPE is a drop-in RoPE augmentation that adds content-aware attention logit biases: a query-gate suppresses irrelevant distant context while a key-gate preserves salient distant tokens. Provably sharper attention and improved long-context robustness — no architecture changes needed.

PolyNSD: Polynomial Neural Sheaf Diffusion

4 minute read

Published:

PolyNSD replaces the NSD propagation operator with a degree-K Chebyshev polynomial in the normalised sheaf Laplacian, achieving SOTA on homo- and heterophilic benchmarks with only diagonal restriction maps and dramatically lower memory usage.

TDA in Materials Science: Topology of Structure and Phase

3 minute read

Published:

Persistent homology characterises the multi-scale structure of materials — from pore geometry in catalysts to glass transition in amorphous materials. H₀ captures connectivity, H₁ captures channels and rings, H₂ captures enclosed voids. These topological descriptors predict mechanical properties, adsorption capacity, and phase transitions more accurately than geometric averages.

TDA in Drug Discovery: Molecular Topology

3 minute read

Published:

Persistent homology provides multi-scale topological fingerprints of molecular structures — capturing rings, cavities, and protein pockets that traditional cheminformatics descriptors miss. Applications include binding site detection, ADMET property prediction, and protein-ligand interaction modelling.