Alessio Borgi

Alessio Borgi

PhD in Graph Neural Networks and Generative AI 👨🏻‍💻🤖

Temporal Knowledge Graphs: Facts That Change Over Time

4 minute read

Published:

TL;DR: A temporal knowledge graph (TKG) extends the standard triple (s, r, o) to a quadruple (s, r, o, t) — each fact has a timestamp or validity interval. TKG completion asks: given (s, r, ?, t), predict the missing entity. This requires reasoning about temporal patterns: periodicity, recency, entity-relation-time interactions.

From Triples to Quadruples

Standard KG: {(s, r, o)} — timeless facts.

Temporal KG: {(s, r, o, t)} where t is a timestamp or interval [t_start, t_end].

Examples:

  • (Barack_Obama, presidentOf, USA, [2009, 2017])
  • (Bayern_Munich, wonChampionsLeague, 2020)
  • (Apple, ceoIs, Steve_Jobs, [1976, 1985] ∪ [1997, 2011])

Two types of TKG facts:

  1. Instantaneous: single timestamp (sports results, news events)
  2. Interval-based: valid during a period (job titles, relationships)

The TKG Completion Task

Interpolation: predict missing facts at known historical times — fill in KG gaps within the training period.

Extrapolation: predict future facts — given everything known up to time t, what triples will be true at t+1?

Extrapolation is the harder and more practically relevant task.

Key Models

TTransE (Time-aware TransE)

Adds time to the TransE scoring function:

f(s, r, o, t) = -||e_s + w_r + w_t - e_o||

Learns a separate time embedding w_t and treats time as another “relation” that shifts entity positions. Simple extension but ignores temporal dynamics.

TNTComplEx

Extends ComplEx to quadruples by adding temporal embeddings as additional mode:

f(s, r, o, t) = Re( e_s · w_r · e_o · w_t )

(4th-order tensor decomposition with complex embeddings.)

RE-NET (Recurrent Event Network)

Uses a recurrent architecture to model temporal sequences:

  1. For each entity pair (s, o), collect the sequence of relations over time
  2. Encode with GRU/LSTM to get history representation
  3. Aggregate with a GNN over the event subgraph at time t
  4. Score candidate triples for t+1

RE-NET explicitly models the temporal ordering of events — capturing recurrence patterns like “Player X scores goals in consecutive matches.”

TGAT (Temporal Graph Attention Network)

Assigns time-encoding to edges and applies attention over temporal neighbourhoods. Each neighbour message is weighted by both structural importance (attention) and temporal proximity (time encoding).

Why temporal patterns matter: "CountryX will hold elections" is more likely if elections occurred ~4 years ago (periodicity). "PersonY will be appointed to a position" depends on whether they recently left another position (temporal sequence). TKG models that capture periodicity and recency dramatically outperform static KG models on extrapolation tasks.

Temporal Reasoning Challenges

1. Irregular observation: facts are not observed at uniform time intervals — some entities have dense histories, others sparse.

2. Time granularity: a fact valid for decades appears at daily/monthly resolution differently than a single-day event.

3. Entity dynamics: entities change identity over time (companies merge, people change roles). The embedding of “CEO of Apple” should change as different people hold the role.

4. Causality vs correlation: temporal patterns in KGs often reflect causal chains, but models learn correlations. Disentangling these is an open problem.

TKG Benchmarks

  • ICEWS (Integrated Crisis Early Warning System): political events worldwide, timestamped daily
  • GDELT: global event database, fine-grained temporal resolution
  • YAGO15K: static YAGO with temporal annotations
  • WikiData (temporal subset): entity facts with validity intervals

Standard splits: train on t ≤ T, validate on T < t ≤ T’, test on t > T’.

Summary

ModelApproachTemporal pattern captured
TTransETime embedding + TransEBasic time displacement
TNTComplEx4-order tensor with timeComplex temporal interactions
RE-NETGNN + RNNTemporal event sequences
TGATTemporal attentionRecency-weighted neighbourhood

Temporal knowledge graphs are a stepping stone from static relational reasoning to full temporal graph learning (covered in the Dynamic Graphs section). The key insight: facts have lifetimes, and reasoning about the world requires reasoning about when facts were true — not just whether they are true.

References

You May Also Enjoy

Open Problems and Future Directions in Sheaf Neural Networks

5 minute read

Published:

Sheaf Neural Networks are a young field with many open research problems. This post surveys the most important unsolved questions: scalability to million-node graphs, stochastic sheaves for uncertainty quantification, sheaf autoencoders for generative models, connections to persistent homology, and unification with other topological deep learning frameworks.

Sheaf GNNs for Molecular Property Prediction

7 minute read

Published:

Molecular graphs are naturally heterophilic — atoms of different types bonded together — and have rich geometric structure (bond angles, torsion angles). Sheaf GNNs can encode this structure via restriction maps that represent the relational geometry between bonded atoms. This post explores how sheaf theory applies to molecular property prediction.

Sheaf GNNs for Heterophilic Node Classification: Benchmarks and Results

6 minute read

Published:

A systematic comparison of sheaf GNNs against baselines on standard heterophilic node classification benchmarks. Which datasets benefit most from sheaf structure? How does stalk dimension d affect performance? When do sheaves win — and when don’t they?

Sheaves Meet Attention: Transformer-Inspired Sheaf Models

6 minute read

Published:

Graph Transformers use global attention to capture long-range dependencies that local message passing misses. Sheaf Transformers combine sheaf-based local aggregation (with learnable relational geometry) with global attention (for long-range dependencies). This post surveys the design space of sheaf-augmented attention models.