Named Entities and Entity Linking: From Spans to Identifiers

A name is not an identifier

“Cambridge” can name several places and appear inside organisation names. Recognising it as a location does not identify which location the writer means. Conversely, “the university” may refer to a known institution without containing its official name.

Keep the representation explicit: a mention has a text span, a contextual type and potentially a linked identifier. A document-level coreference system groups mentions that refer to the same discourse entity; linking connects that entity or mention to an external inventory. These tasks overlap but are not interchangeable.

Detection has structural choices

A BIO tagger naturally represents non-overlapping contiguous spans. “University of North Harbour” may contain a location mention nested inside an organisation mention, which flat BIO cannot encode simultaneously. Discontinuous mentions need another representation again.

Decide whether the annotation includes nesting, adjectival mentions, metonymy and generic references. “London announced a policy” may refer to a government through a place name. The correct type depends on the annotation policy, not solely on the spelling.

Retrieve candidates before ranking them

Candidate generation uses aliases, lexical search, redirects, transliteration or learned retrieval. The candidate set must contain the correct entity before a reranker can select it. If gold-entity recall@10 is 0.90, no reranker restricted to those candidates can exceed 0.90 linking accuracy on that evaluated set.

A conditional pipeline calculation makes the bottleneck visible. Suppose mention detection recall is 0.90, candidate recall given a correct detected mention is 0.95, and ranking accuracy given both earlier successes is 0.80. Correct end-to-end recovery is \(0.90\times0.95\times0.80=0.684\) under those conditional rates. This is a chain-rule calculation, not an independence assumption.

Score contextual compatibility

A simple ranking score may combine a mention prior and context similarity:

Here \(m\) is the surface mention, \(c\) its context, \(v_e\) an entity description vector and \(f\) additional features. This is a score, not automatically a calibrated posterior. A frequent entity prior helps on common senses but can overwhelm evidence for a rare entity.

For “Cambridge won the boat race”, a place-popularity prior may be misleading; the event context supports a university or team interpretation. Read enough surrounding text to resolve the task, and avoid using document metadata that will not exist at deployment.

NIL is a real outcome

A mention may refer to an entity absent from the knowledge base. Forcing every mention to the nearest candidate creates confident wrong identifiers. Include a NIL or abstention decision and tune it with examples of entities genuinely absent from the indexed inventory.

A knowledge-base snapshot is part of the experiment. An entity missing from an old snapshot may be present in a later one; aliases and descriptions also change. Report the snapshot and candidate-generation resources so another run uses the same task.

Evaluate the stages and the whole pipeline

Use exact-span/type precision and recall for NER, candidate recall for retrieval, and linking accuracy conditioned on gold mentions to isolate disambiguation. Then evaluate the complete predicted-span-plus-ID output. A system can improve oracle-mention linking while getting worse end to end if detection changes.

Inspect errors on rare entities, ambiguous aliases, nested names and out-of-inventory mentions. Also inspect provenance: a correct identifier attached to the wrong text span may still produce a bad extraction record. Store the source offsets so the user can verify the association in context.

References

1. Tjong Kim Sang and De Meulder. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. 2003.
2. Hoffart et al. Robust Disambiguation of Named Entities in Text. EMNLP, 2011.