Alessio Borgi

Alessio Borgi

PhD in Graph Neural Networks and Generative AI 👨🏻‍💻🤖

GNNs for Recommender Systems

7 minute read

Published:

TL;DR: The user-item interaction graph is bipartite: users on one side, items on the other, with edges representing clicks, purchases or ratings. GCN-style propagation on this graph captures multi-hop collaborative signals — "users who liked what you liked also liked X." LightGCN strips the layer down to pure normalised propagation, dropping both the weight matrix and the non-linearity, and reports better accuracy and fewer parameters than the NGCF design it ablates.
PinSage recommendation GNN
PinSage: graph convolutional network for web-scale recommender systems (Ying et al., 2018)

Recommendation as a Graph Problem

Intuition First: Matrix factorisation is like learning that “Alice likes comedies” and “this film is a comedy” and multiplying those two vectors. It captures direct user–item similarity but cannot represent the chain: “Alice liked this film, Bob also liked it, Bob also liked that other film, so Alice might like that other film too.” GNNs capture this multi-hop chain by propagating information along the bipartite graph — 2-hop neighbours of Alice (items liked by users who liked Alice’s items) are exactly the collaborative filtering signal that matrix factorisation misses.

Traditional collaborative filtering: learn a user embedding \(e_u\) and an item embedding \(e_i\), then predict the score as \(\hat{y}_{ui} = e_u^{\top} e_i\). This captures pairwise similarity but not higher-order structure.

GNN approach: build a bipartite graph in which user \(u\) is joined to item \(i\) whenever \(u\) interacted with \(i\). Run a GNN to produce embeddings that capture multi-hop neighbourhood structure:

  • 1-hop: items \(u\) has interacted with (or users who interacted with \(i\))
  • 2-hop: items interacted with by users who also interacted with \(u\)’s items — the collaborative filtering signal
  • 3-hop: transitive similarities

The Bipartite User-Item Graph

Let \(U\) be the users and \(I\) the items, with \(V = U \sqcup I\):

\[ G = (U \sqcup I,\; E), \qquad (u, i) \in E \iff \text{user } u \text{ interacted with item } i . \]

Because the graph is bipartite, message passing alternates sides — users only ever hear from items, and items only from users:

\[ h_u^{(k)} = \mathrm{AGG}\!\left( \{\, h_i^{(k-1)} : i \in \mathcal{N}(u) \,\} \right), \qquad h_i^{(k)} = \mathrm{AGG}\!\left( \{\, h_u^{(k-1)} : u \in \mathcal{N}(i) \,\} \right). \]

After \(K\) layers, \(h_u^{(K)}\) encodes the \(K\)-hop neighbourhood. Note the parity: odd \(k\) mixes in items, even \(k\) mixes in other users, so it takes at least two layers before any collaborative signal reaches \(u\) at all.

LightGCN (He et al., 2020)

LightGCN makes a key simplification: remove the weight matrices and the non-linearities. Each layer is nothing but symmetrically normalised averaging over the bipartite graph:

\[ h_u^{(k)} = \sum_{i \in \mathcal{N}(u)} \frac{1}{\sqrt{\lvert \mathcal{N}(u) \rvert \, \lvert \mathcal{N}(i) \rvert}}\, h_i^{(k-1)}, \qquad h_i^{(k)} = \sum_{u \in \mathcal{N}(i)} \frac{1}{\sqrt{\lvert \mathcal{N}(i) \rvert \, \lvert \mathcal{N}(u) \rvert}}\, h_u^{(k-1)}. \]

The final embedding is a weighted combination of every layer, which is how the model retains the 0-hop signal instead of over-smoothing it away:

\[ e_u = \sum_{k=0}^{K} \alpha_k\, h_u^{(k)}, \qquad \alpha_k = \frac{1}{K+1}, \qquad \hat{y}_{ui} = e_u^{\top} e_i . \]

Why remove the transformations? Because on these benchmarks there is nothing for them to transform. The paper’s ablation removes \(W_k\) and \(\sigma(\cdot)\) one at a time and finds that each removal helps, with the combined removal helping most: the collaborative signal lives in the propagation, and extra learnable matrices mainly add capacity to overfit.

LightGCN's key insight: Standard GCNs were designed for graphs with rich node features, where \(W\) does real work reprojecting them. In pure collaborative filtering the only input is a free ID embedding, already learned end to end — so \(W\) merely reparameterises a vector the model was free to choose anyway, buying no expressiveness while adding parameters and an optimisation burden. Stripping it back leaves the normalised propagation, which is the part that actually carries collaborative signal. This is a recommender-specific argument, not a general claim that simpler GNNs are better.

PinSage (Ying et al., 2018)

Pinterest’s GNN for image recommendation, and one of the first published industrial deployments of a GNN.

Scale: the paper reports a bipartite pin–board graph of 3 billion nodes and roughly 18 billion edges.

Key innovations:

  1. GraphSAGE-style sampling: for each node, sample a fixed-size neighbourhood rather than the full one — this is what makes the computation tractable, since a full-neighbourhood pass over a graph this size is hopeless
  2. Random-walk importance sampling: choose neighbours by how often they are visited in short random walks from the target node, not uniformly, and weight their messages by that visit count
  3. Curriculum training: feed progressively harder negative examples as training proceeds

NGCF and Variants

NGCF (Wang et al., 2019): adds an explicit feature interaction to the message:

\[ m_{u \leftarrow i} = \frac{1}{\sqrt{\lvert \mathcal{N}(u) \rvert \, \lvert \mathcal{N}(i) \rvert}} \Big( W_1 h_i + W_2 \left( h_i \odot h_u \right) \Big). \]

The Hadamard product \(h_i \odot h_u\) is meant to capture user-item feature interaction. LightGCN’s ablation found that on the standard collaborative-filtering benchmarks this term costs more in overfitting than it returns in expressiveness — though the argument is specific to the ID-embedding-only setting, and richer side features change the calculus.

2-hop Collaborative Filtering via GNN Alice Bob Item1 Item2 Item3 1-hop: items Alice liked Recommend Item3 to Alice 2-hop path: Alice – Item2 – Bob – Item3 (dashed amber = predicted link)
GNN propagation on the bipartite graph: Alice and Bob both liked Item2 (1-hop, solid edges). Bob also liked Item3, so Item3 is two hops from Alice. LightGCN propagates the signal along that path and scores the missing Alice–Item3 edge (dashed amber) highly.

Session-Based Recommendation

Standard CF assumes all past interactions are known for each user. Session-based recommendation has no long-term user history — only the current session (sequence of clicks).

SR-GNN (Wu et al., 2019): model a session as a directed graph (clicks are edges from previous item to next item). Run GCN on session graph, then use attention to extract user intent from node embeddings. This captures transition patterns between items within a session.

Knowledge Graph-Enhanced Recommendation

KGNN-LS / KGCN: enrich the item side with a knowledge graph (item → category, brand, attributes). GNN propagates over both the user-item graph and the item knowledge graph simultaneously.

Benefit: cold-start items with no interactions can leverage KG features (genre, director for movies) to receive recommendations from users with similar taste in KG-related items.

Summary

ModelKey ideaHops used
Matrix factorisationPairwise similarity only0 (direct inner product)
NGCFGCN + explicit feature interaction\(K\) hops, with \(W_1, W_2\) per layer
LightGCNNormalised propagation, no \(W\) or \(\sigma\)\(K\) hops, layer-combined
PinSageGraphSAGE + random-walk importance samplingSampled 2-hop neighbourhoods
SR-GNNSession graph + gated GNNWithin-session transitions

The mechanism that unifies these is the one worth remembering: on a bipartite interaction graph, a \(K\)-layer GNN gives every user an embedding that already contains the items liked by users like them. That is collaborative filtering expressed as message passing, and it is why PinSage could be deployed at Pinterest’s graph scale while a dense user-item matrix could not.

References

You May Also Enjoy

The Standard Library Modules Worth Knowing Before You pip install

8 minute read

Published:

Python ships with a counter, a queue, a memoiser, a CLI parser, a test framework’s smaller cousin, and a JSON-safe date library already installed — most of what people reach for a package to solve is one import away.

Files and Context Managers: Why with Is Not Optional

9 minute read

Published:

A file object that falls out of scope without with does get closed eventually — but ‘eventually’ means whenever the garbage collector gets around to it, which is not a promise any program handling more than a handful of files can live with.

Errors and Exceptions: EAFP, the Hierarchy, and Reading a Traceback

9 minute read

Published:

Exceptions in Python are not exceptional. They are a normal control-flow mechanism that the language leans on so heavily that the idiomatic style is to try the operation and handle the failure, rather than check first — which is faster, shorter, and free of race conditions.