Alessio Borgi

Alessio Borgi

PhD in Graph Neural Networks and Generative AI 👨🏻‍💻🤖

HAN: Heterogeneous Graph Attention Networks

6 minute read

Published:

TL;DR: HAN (Wang et al., 2019) uses pre-defined meta-paths to decompose a heterogeneous graph into multiple homogeneous views. GAT-style attention aggregates neighbours within each view (node-level attention), and a second attention layer combines the meta-path views (semantic-level attention). The model learns both which neighbours matter and which relationships matter.
HAN architecture
Heterogeneous Attention Network (HAN): node-level and semantic-level attention (Wang et al., 2019)

The Two-Level Attention Idea

Key Insight: HAN asks two questions simultaneously: (1) "Among all co-authors of paper P, which ones are most relevant?" — that is node-level attention. (2) "For classifying this author's research area, is co-authorship more informative than same-venue publication?" — that is semantic-level attention. Most models answer only one of these; HAN answers both in a single end-to-end framework.

HAN’s key insight: in a heterogeneous graph, not all neighbours are equally informative, and not all relationship types are equally relevant to the task. Two separate attention mechanisms handle these two sources of variability.

  • Node-level attention: given a meta-path \(\Phi\), weight the importance of the different neighbours reachable via \(\Phi\)
  • Semantic-level attention: weight the importance of the different meta-paths \(\Phi_1, \dots, \Phi_P\) for the overall prediction

The order matters and is not arbitrary: node-level attention runs first, producing one embedding per node per meta-path, and semantic-level attention then runs over those to fuse the meta-path views into a single embedding. The two levels operate on different objects — neighbours within a view, then views themselves.

Step 1: Meta-Path-Based Neighbour Projection

For each meta-path \(\Phi\), the heterogeneous graph induces a projected meta-path graph: a homogeneous graph in which an edge connects \(u\) to \(v\) whenever some path of type \(\Phi\) exists between them.

For example, in an academic network:

  • APA (\(\text{Author} \to \text{Paper} \to \text{Author}\)): co-authorship graph
  • APCPA (\(\text{Author} \to \text{Paper} \to \text{Conference} \to \text{Paper} \to \text{Author}\)): same-venue collaboration graph

Each meta-path gives a different view of the Author–Author relationships. Because these views are built by composing edges, they are typically far denser than the original graph — the APA neighbourhood of a prolific author can be very large — which is one source of HAN’s cost.

Before attention, project all node features to a common dimension (different node types may arrive with different feature dimensions):

\[ h'_v \;=\; \sigma\bigl(W_{\tau(v)}\, h_v\bigr) \]

Step 2: Node-Level Attention (Within Meta-Path)

For a given meta-path \(\Phi\), apply GAT-style attention over the meta-path neighbourhood \(\mathcal{N}^{\Phi}_i\).

Compute an unnormalised attention score between nodes \(i\) and \(j\):

\[ e^{\Phi}_{ij} \;=\; \sigma\Bigl(\bigl(a^{\Phi}\bigr)^{\top} \bigl[\, h'_i \,\Vert\, h'_j \,\bigr]\Bigr) \]

Here \(a^{\Phi} \in \mathbb{R}^{2d'}\) is a meta-path-specific attention vector and \(\Vert\) denotes concatenation. Crucially \(a^{\Phi}\) is not shared between meta-paths: what makes a neighbour important along a co-authorship path is a different question from what makes one important along a same-venue path.

Normalise over the meta-path neighbourhood with a softmax:

\[ \alpha^{\Phi}_{ij} \;=\; \frac{\exp\bigl(e^{\Phi}_{ij}\bigr)}{\sum_{k \in \mathcal{N}^{\Phi}_i} \exp\bigl(e^{\Phi}_{ik}\bigr)} \]

Aggregate:

\[ z^{\Phi}_i \;=\; \sigma\Bigl(\sum_{j \in \mathcal{N}^{\Phi}_i} \alpha^{\Phi}_{ij}\, h'_j\Bigr) \]

\(z^{\Phi}_i\) is node \(i\)’s embedding under meta-path \(\Phi\). With multi-head attention, run \(K\) independent heads and concatenate them:

\[ z^{\Phi}_i \;=\; \Big\Vert_{k=1}^{K} \; \sigma\Bigl(\sum_{j \in \mathcal{N}^{\Phi}_i} \alpha^{\Phi,k}_{ij}\, h'_j\Bigr) \]

Step 3: Semantic-Level Attention (Across Meta-Paths)

After computing \(\{z^{\Phi_1}_i, \dots, z^{\Phi_P}_i\}\) for every node \(i\), we need to combine them, because each meta-path may contribute differently to the task.

Learn a meta-path importance score. Note it is shared across nodes: a meta-path’s relevance is treated as a global property of the task, so the score is averaged over all nodes before the softmax rather than computed per node.

\[ w^{\Phi_p} \;=\; \frac{1}{\lvert V \rvert} \sum_{i \in V} q^{\top} \tanh\bigl(W z^{\Phi_p}_i + b\bigr) \]

Normalise across the \(P\) meta-paths:

\[ \beta^{\Phi_p} \;=\; \frac{\exp\bigl(w^{\Phi_p}\bigr)}{\sum_{p'=1}^{P} \exp\bigl(w^{\Phi_{p'}}\bigr)} \]

Final embedding — the same \(P\) weights applied to every node:

\[ h_i \;=\; \sum_{p=1}^{P} \beta^{\Phi_p}\, z^{\Phi_p}_i \]

This is the design decision that distinguishes the two levels. Node-level attention is per node: node \(i\) decides which of its own neighbours matter. Semantic-level attention is per graph: one set of \(P\) weights, learned from the whole node set, decides which relationships matter for the task. The consequence is a limitation as well as a simplification — HAN cannot let one author be better explained by co-authorship while another is better explained by venue.

What semantic attention learns: If the task is author classification (research area prediction), APA (co-authorship) may get a high \(\beta\) — co-authors usually work in the same area. APCPA (same venue) may get a lower \(\beta\) — researchers can publish in the same venue across areas. HAN learns this from labelled data alone, without manual meta-path weighting.

Full HAN Pipeline

Input: Heterogeneous graph G with multiple node/edge types
       Pre-defined meta-paths {Φ₁, ..., Φ_P}

For each meta-path Φ_p:
  1. Project node features to common space
  2. Construct meta-path graph (who is connected via Φ_p?)
  3. Apply node-level GAT attention → z^{Φ_p}_i for each node

Semantic-level attention:
  4. Compute meta-path importance β^{Φ_p}
  5. Weighted combination: h_i = Σ_p β^{Φ_p} z^{Φ_p}_i

Classifier:
  6. MLP(h_i) → ŷ_i

Worked Example: Semantic Attention Weights

Suppose we have two meta-paths for author classification:

  • APA (\(\text{Author} \to \text{Paper} \to \text{Author}\)): co-authorship
  • APCPA (\(\text{Author} \to \text{Paper} \to \text{Conference} \to \text{Paper} \to \text{Author}\)): same-venue collaboration

After node-level attention, author \(i\) has two embeddings, \(z^{\mathrm{APA}}_i\) and \(z^{\mathrm{APCPA}}_i\).

Semantic attention computes the two importance scores, each averaged over all authors:

w_APA   = (1/|V|) Σ_i  qᵀ · tanh(W · z^APA_i   + b)  →  suppose  w_APA   = 0.8
w_APCPA = (1/|V|) Σ_i  qᵀ · tanh(W · z^APCPA_i + b)  →  suppose  w_APCPA = 0.3

After softmax normalisation over the two meta-paths:

\(\beta_{\mathrm{APA}} = \dfrac{e^{0.8}}{e^{0.8} + e^{0.3}} \approx \dfrac{2.23}{2.23 + 1.35} \approx 0.62\), and \(\beta_{\mathrm{APCPA}} \approx \dfrac{1.35}{3.58} \approx 0.38\).

Final embedding: \(h_i = 0.62\, z^{\mathrm{APA}}_i + 0.38\, z^{\mathrm{APCPA}}_i\) — the same two weights for every author in the graph.

The model has learned that co-authorship carries more weight than shared venue for research area prediction, and it learned it from the labels alone rather than from a hand-set weighting. Read the ratio with care, though: \(\beta\) weights are relative to the meta-path set you chose and to the scale of the \(z^{\Phi}\) vectors, so they rank the views you supplied — they are not a calibrated measure of how informative each relationship is in absolute terms.

Limitations

  1. Manual meta-path definition: domain knowledge required to define meaningful meta-paths. Wrong meta-paths → poor performance.

  2. Combinatorial explosion: for complex HINs with many relation types, the number of meaningful meta-paths grows combinatorially.

  3. Node-type coverage: HAN focuses on target node type; other node types are only intermediaries in meta-paths.

  4. Scalability: constructing meta-path graphs and running multiple GATs is expensive for large graphs.

Later architectures (HGT: Heterogeneous Graph Transformer) address these by learning relation-specific attention without explicit meta-path definition.

Summary

ComponentWhat it learnsScope
Node projection \(W_{\tau(v)}\)Common embedding space across typesPer node type
Node-level attention \(\alpha^{\Phi}_{ij}\)Which neighbours matterPer node, per meta-path
Semantic attention \(\beta^{\Phi_p}\)Which meta-paths matterPer graph (shared by all nodes)
Multi-head node attentionMultiple perspectives within a meta-pathPer node, per meta-path

HAN applies attention at two levels to heterogeneous graphs, making the point that which relation is as much a learnable question as which neighbour in complex multi-relational data.

References

You May Also Enjoy

The Standard Library Modules Worth Knowing Before You pip install

8 minute read

Published:

Python ships with a counter, a queue, a memoiser, a CLI parser, a test framework’s smaller cousin, and a JSON-safe date library already installed — most of what people reach for a package to solve is one import away.

Files and Context Managers: Why with Is Not Optional

9 minute read

Published:

A file object that falls out of scope without with does get closed eventually — but ‘eventually’ means whenever the garbage collector gets around to it, which is not a promise any program handling more than a handful of files can live with.

Errors and Exceptions: EAFP, the Hierarchy, and Reading a Traceback

9 minute read

Published:

Exceptions in Python are not exceptional. They are a normal control-flow mechanism that the language leans on so heavily that the idiomatic style is to try the operation and handle the failure, rather than check first — which is faster, shorter, and free of race conditions.