Alessio Borgi

Alessio Borgi

PhD in Graph Neural Networks and Generative AI 👨🏻‍💻🤖

Temporal Sheaves: Extending Sheaf GNNs to Dynamic Graphs

6 minute read

Published:

TL;DR: A temporal sheaf F_t assigns stalks and restriction maps that vary continuously with time t. The sheaf Laplacian Δ_{F_t} is itself time-dependent — encoding the evolving relational geometry of the graph. The natural dynamics are given by a sheaf ODE: dH/dt = −Δ_{F_t} H, where both H (node states) and F_t (restriction maps) evolve. This subsumes both standard graph neural ODEs (when F_t = I) and TGN (when maps are updated by a memory module).

Why Temporal Sheaves?

Standard sheaf GNNs are defined on static graphs. Real-world graphs are dynamic:

  • Social networks: friendships are added and removed over time
  • Citation networks: papers appear sequentially
  • Biological networks: protein interactions change with cellular context
  • Financial networks: transactions happen at specific times

A temporal sheaf models two sources of dynamics:

  1. Structural dynamics: the graph topology G_t changes (new nodes, new edges)
  2. Map dynamics: the restriction maps F_t change even when topology is fixed (the relational geometry evolves)

The Temporal Sheaf Framework

A temporal sheaf over time T = [0, ∞) assigns:

  • For each t ∈ T: a graph G_t = (V_t, E_t)
  • For each node v and time t: a stalk F_t(v) ≅ ℝ^d
  • For each edge e ∈ E_t and time t: a stalk F_t(e) ≅ ℝ^d
  • For each incident pair (v, e) at time t: a restriction map F_{t,v▷e} : F_t(v) → F_t(e)

The time-dependent Sheaf Laplacian Δ_{F_t} is built from the maps at time t.

Two natural dynamics:

  • Discrete temporal sheaf: G_t is piecewise constant, changing at event times t₁, t₂, …. The Sheaf Laplacian jumps at each event.
  • Continuous temporal sheaf: G_t and F_t evolve continuously. The Sheaf Laplacian is Lipschitz-continuous in t.

Sheaf ODEs: Continuous-Time Dynamics

The natural continuous-time dynamics on a temporal sheaf is the sheaf heat equation:

dH(t)/dt = −Δ_{F_t} H(t) , H(0) = X₀

where H(t) ∈ ℝ^{Nd} is the time-varying node stalk signal and Δ_{F_t} is the time-varying Sheaf Laplacian.

This is a time-varying linear ODE — the coefficient matrix changes with time. Unlike the static case (constant Δ_F), the solution is not simply exp(−Δ_F t)X₀. Instead, it is given by the time-ordered exponential (Magnus expansion):

H(t) = T{exp(−∫₀^t Δ_{F_s} ds)} H(0)

For slowly varying maps, the adiabatic approximation applies: H(t) ≈ proj_{H⁰(G_t, F_t)} H(0) + corrections from the map velocity dF_t/dt.

Coupling Map Learning with State Evolution

In a learnable temporal sheaf GNN, the restriction maps and node states evolve jointly:

dH(t)/dt = −Δ_{F_t} H(t) (state diffuses along sheaf) dF_t/dt = g(H(t), F_t) (maps evolve based on current state)

This is a coupled ODE system — the maps drive the diffusion, and the diffusion updates the states that drive the map updates.

In practice, this is implemented as an alternating scheme:

  1. Given H^{(k)} and F^{(k)}, compute Δ_{F^{(k)}} and update H^{(k+1)} via one diffusion step
  2. Given H^{(k+1)}, update the maps F^{(k+1)} via the sheaf predictor MLP

This is equivalent to a recurrent sheaf architecture where the restriction maps are re-predicted at each timestep.

Connection to TGN: In Temporal Graph Networks (Rossi et al., 2020), a memory module s_v(t) encodes the interaction history of each node, and an embedding module uses s_v(t) to compute node representations. The temporal sheaf framework provides a principled interpretation: s_v(t) is the node stalk state H_v(t), and the memory update is the sheaf ODE. The TGN "message" from u to v at time t corresponds to F_{t,u▷e}ᵀ F_{t,v▷e} H_u(t) — the transported stalk from u to v via the current restriction maps.

Event-Driven Sheaf Updates

For continuous-time dynamic graphs (CTDG) where interactions happen at discrete events:

At event (u, r, v, t):

  1. Update restriction map: F_{t,u▷e} and F_{t,v▷e} are recomputed via MLP(H_u(t), H_v(t))
  2. Update Δ_{F_t}: only the blocks involving edge e change — O(d²) local update
  3. Update states: run a brief ODE integration step (or single Euler step) to propagate the event’s effect

This event-driven scheme is analogous to TGN’s message-passing update but with sheaf structure.

Snapshot Sheaf GNNs for DTDG

For discrete-time dynamic graphs (DTDG) where the graph is observed as snapshots G₁, G₂, …, G_T:

Define a snapshot sheaf as a sequence (F₁, F₂, …, F_T) of static sheaves, with:

  • Per-snapshot sheaf diffusion: H^{(k)}{t+1} = (I − Δ{F_t}^{norm}) H^{(k)}_t W^{(k)}_t
  • Cross-snapshot recurrence: H^{(0)}_{t+1} = GRU(H^{(K)}_t, H^{(0)}_t) or LSTM update

This is exactly DCRNN or STGCN with the graph Laplacian replaced by the Sheaf Laplacian — a natural generalisation for heterophilic dynamic graphs.

Temporal Restriction Maps: What They Capture

In a temporal sheaf, the restriction maps F_{t,v▷e} change with time. What does this mean?

Map drift: If F_{t,v▷e} → F_{t’,v▷e} as t → t’, the relational geometry of edge e is gradually shifting. This could represent:

  • A social relationship changing in nature (from acquaintance to close friend)
  • A protein interaction changing affinity with cellular context
  • A financial transaction pattern evolving with market conditions

Map volatility: High-frequency oscillations in F_t can represent oscillatory relational patterns — seasonal effects, periodic interactions.

Map discontinuities: Sudden jumps in F_t at events represent abrupt relational changes — a breakup of a social connection, a company merger.

Temporal Sheaf Benchmarks

Existing temporal GNN benchmarks (JODIE, TGN, TGAT):

  • Reddit: 10,984 nodes, 672,447 timestamped interactions
  • MOOC: student-course interactions with timestamps
  • LastFM: music listening events

These benchmarks test link prediction under the CTDG setting. Temporal sheaf GNNs can be evaluated on these by replacing the standard GNN component with sheaf diffusion, while retaining the memory module for long-range history.

Expected benefit: On heterophilic dynamic graphs (e.g., academic citation graphs where papers cite papers of different fields), temporal sheaf GNNs should outperform standard TGN by correctly handling the heterophilic relational structure at each timestep.

Open Questions

  1. Map memory: Should the restriction maps themselves be stored in the memory module? This would give maps that evolve with the interaction history, not just the current features.
  2. Temporal H⁰: The global section space H⁰(G_t, F_t) evolves with time — tracking its evolution provides a topological time series of the graph’s relational structure.
  3. Temporal sheaf Laplacian spectrum: How do eigenvalues of Δ_{F_t} evolve? Do they exhibit phase transitions at critical graph changes?

References

  • Rossi, E., Chamberlain, B., Frasca, F., Eynard, D., Monti, F., & Bronstein, M. (2020). Temporal Graph Networks for Deep Learning on Dynamic Graphs. ICML GRL+ Workshop 2020 (TGN: memory-based continuous-time GNN — the architecture that temporal sheaves generalise).
  • Poli, M., Massaroli, S., Park, J., Yamashita, A., Asama, H., & Park, J. (2019). Graph Neural Ordinary Differential Equations. arXiv 2019 (Graph Neural ODE: continuous-time graph dynamics — the ODE framework that sheaf ODEs extend with sheaf structure).
  • Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018). Neural Ordinary Differential Equations. NeurIPS 2018 (Neural ODEs: the adjoint method for continuous ODE systems — foundation for sheaf ODE training).

You May Also Enjoy

Open Problems and Future Directions in Sheaf Neural Networks

5 minute read

Published:

Sheaf Neural Networks are a young field with many open research problems. This post surveys the most important unsolved questions: scalability to million-node graphs, stochastic sheaves for uncertainty quantification, sheaf autoencoders for generative models, connections to persistent homology, and unification with other topological deep learning frameworks.

Sheaf GNNs for Molecular Property Prediction

7 minute read

Published:

Molecular graphs are naturally heterophilic — atoms of different types bonded together — and have rich geometric structure (bond angles, torsion angles). Sheaf GNNs can encode this structure via restriction maps that represent the relational geometry between bonded atoms. This post explores how sheaf theory applies to molecular property prediction.

Sheaf GNNs for Heterophilic Node Classification: Benchmarks and Results

6 minute read

Published:

A systematic comparison of sheaf GNNs against baselines on standard heterophilic node classification benchmarks. Which datasets benefit most from sheaf structure? How does stalk dimension d affect performance? When do sheaves win — and when don’t they?

Sheaves Meet Attention: Transformer-Inspired Sheaf Models

6 minute read

Published:

Graph Transformers use global attention to capture long-range dependencies that local message passing misses. Sheaf Transformers combine sheaf-based local aggregation (with learnable relational geometry) with global attention (for long-range dependencies). This post surveys the design space of sheaf-augmented attention models.