Temporal Knowledge Graphs: Facts That Change Over Time

From Triples to Quadruples

Standard KG: \(\{(s, r, o)\}\) — timeless facts.

Temporal KG: \(\{(s, r, o, t)\}\) where \(t\) is a timestamp or an interval \([t_{\text{start}}, t_{\text{end}}]\).

Examples:

• (Barack Obama, presidentOf, USA, [2009, 2017])
• (Bayern Munich, wonChampionsLeague, 2020)
• (Apple, ceoIs, Steve Jobs, [1997, 2011])

Two types of TKG facts:

1. Instantaneous: single timestamp (sports results, news events)
2. Interval-based: valid during a period (job titles, relationships)

The TKG Completion Task

Interpolation: predict missing facts at known historical times — fill in KG gaps within the training period.

Extrapolation: predict future facts — given everything known up to time t, what triples will be true at t+1?

Extrapolation is the harder and more practically relevant task.

Key Models

TTransE (Time-aware TransE)

Adds time to the TransE scoring function as a second translation:

Each timestamp gets its own embedding \(w_t\), and time acts as another “relation” that shifts entity positions. It is the simplest possible temporal extension, and it inherits both TransE’s limits (no symmetric relations) and one of its own: \(w_t\) is a lookup table over observed timestamps, so a timestamp never seen in training has no embedding. That makes plain TTransE an interpolation model — extrapolating to future times needs either a parametric function of \(t\) or a model that reasons over history.

TComplEx / TNTComplEx

Extends ComplEx to quadruples by treating time as a fourth mode of the tensor:

This 4th-order decomposition with complex embeddings is TComplEx. TNTComplEx (“temporal and non-temporal”) adds a second, time-independent ComplEx term to the score, so a relation that never changes — bornIn — is modelled by the static part rather than being forced to reproduce itself across every timestamp. Splitting the two is the paper’s main contribution over the plain temporal factorisation.

RE-NET (Recurrent Event Network)

Models the sequence of a subject’s past events autoregressively:

1. For a subject \(s\) (optionally conditioned on a relation \(r\)), collect its events grouped by timestamp
2. At each past timestep, a neighbourhood aggregator summarises the set of concurrent events involving \(s\) into one vector
3. An RNN encodes that sequence of per-timestep summaries into a history representation
4. Score candidate objects for the next timestep from the history

The ordering of events is what the RNN consumes, so RE-NET can capture recurrence and sequence patterns — “Player X scores in consecutive matches”, or a state visit following a state visit — that a model treating each timestamp independently cannot.

TGAT (Temporal Graph Attention Network)

Attaches a functional time encoding to each edge and applies attention over temporal neighbourhoods, so each neighbour message is weighted by both structural importance (attention) and temporal proximity. The word “functional” carries the weight here: TGAT’s encoding is a continuous function of the elapsed time rather than a lookup table over observed timestamps, so unlike TTransE it can be evaluated at a time it has never seen. That is what makes it usable for extrapolation.

Worked Example: TTransE on a Political Event

Suppose we want to predict \((\text{CountryX},\ \textit{holdsElection},\ ?,\ t = 2024)\).

Static TransE embeds CountryX and holdsElection with no notion of time — its score is the same for every year, so it either always ranks an election highly or never does, according to training frequency.

TTransE scores with \(f(s, r, o, t) = -\lVert e_s + w_r + w_t - e_o\rVert\). If training has driven \(w_{2024}\) close to \(w_{2020}\) — both election years, the same phase of a four-year cycle — then 2024 candidates score much as 2020’s did, and the periodicity is captured. A non-election year like 2022 has a \(w_{2022}\) far from \(w_{2020}\), so election predictions there score low.

Two caveats keep this honest. First, nothing in TTransE makes \(w_{2024}\) resemble \(w_{2020}\) — the time embeddings are free parameters, and they only end up near each other if enough training quadruples pull them there. Second, and more restrictive, \(w_{2024}\) has to exist at all: if 2024 never appears in training, TTransE has no embedding for it and cannot score the query. Genuine extrapolation therefore needs either time embeddings that are a smooth function of \(t\) (so unseen timestamps can be evaluated) or a history-based model like RE-NET that conditions on the past rather than looking up the future.

Temporal Reasoning Challenges

1. Irregular observation: facts are not observed at uniform time intervals — some entities have dense histories, others sparse.

2. Time granularity: a fact valid for decades appears at daily/monthly resolution differently than a single-day event.

3. Entity dynamics: entities change identity over time (companies merge, people change roles). The embedding of “CEO of Apple” should change as different people hold the role.

4. Causality vs correlation: temporal patterns in KGs often reflect causal chains, but models learn correlations. Disentangling these is an open problem.

TKG Benchmarks

• ICEWS (Integrated Crisis Early Warning System): political events worldwide, timestamped daily
• GDELT: global event database, fine-grained temporal resolution
• YAGO15K: static YAGO with temporal annotations
• WikiData (temporal subset): entity facts with validity intervals

Standard splits are chronological, not random: train on \(t \le T\), validate on \(T < t \le T'\), test on \(t > T'\). A random split would leak future information into training and inflate results.

Summary

Temporal knowledge graphs are a stepping stone from static relational reasoning to full temporal graph learning (covered in the Dynamic Graphs section). The key insight: facts have lifetimes, and reasoning about the world requires reasoning about when facts were true — not just whether they are true.

References

• Lacroix, T., Obozinski, G., & Usunier, N. (2020). Tensor Decompositions for Temporal Knowledge Base Completion. ICLR 2020 (TNTComplEx).
• Jin, W., Qu, M., Jin, X., & Ren, X. (2020). Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs. EMNLP 2020 (RE-NET).
• Xu, D., Ruan, C., Körpeoglu, E., Kumar, S., & Achan, K. (2020). Inductive Representation Learning on Temporal Graphs. ICLR 2020 (TGAT).