A Primer on Ontology Matching

New to the field? This page explains what the track asks systems to do, then lets you try both tasks by hand on toy examples.

What is an ontology?

An ontology is a formal, machine-readable vocabulary for a domain: its classes (kinds of things, e.g. TemperatureSensor), its properties (relations and attributes, e.g. isLocatedIn), and the logical axioms connecting them. Two organisations modelling the same world almost never produce the same ontology — names, granularity, and structure all differ.

What is ontology matching?

Ontology matching (or alignment) is the task of automatically finding which entities in one ontology correspond to which entities in another. A single finding is a correspondence:

⟨ entity₁, entity₂, relation, confidence ⟩ — for example ⟨ thinkhome:CoolCommand, brick:Cooling_Command, =, 1.0 ⟩.

A set of correspondences between two ontologies is an alignment. Matching systems are evaluated by comparing their alignment against a reference alignment curated by experts. DISO-OAEI's references are partial but locally complete "silver standards": for every entity they mention, they list all of that entity's correct correspondences — but they are silent about unmentioned entities. The scoring accounts for this (that's the "partiality-corrected" in our metrics: predictions the reference cannot adjudicate don't count against you).

Try it — Task 1 in miniature

Two toy ontologies. Select one entity on each side to propose a correspondence, then watch precision, recall and F1 respond. The hidden reference covers only the sensor / heating / window entities — Room ↔ Space is a perfectly good mapping the silver simply doesn't mention, so proposing it neither helps nor hurts. That is partiality correction at work.

The interactive demo needs JavaScript. A worked example: proposing TemperatureSensor↔Temperature_Sensor and Heater↔Lighting_System gives Â = 2 judgeable proposals, 1 correct: P = 1/2 = 0.50, R = 1/3 ≈ 0.33, F1 = 0.40.

HomeOnt (source)

BuildOnt (target)

When there is no right answer: ranking with NIL

Task 2 flips the problem around. Instead of generating an alignment, a system receives, for each query (a source entity), a fixed pool of 50 candidate targets and must order the pool best-first. One special candidate — NIL — means "no equivalent exists." Ranking NIL first is how a system abstains. Scoring simply asks: where in your ranking did the right answer land? Hits@k is the fraction of queries whose right answer is in the top k; MRR averages 1/rank.

Try it — Task 2 in miniature

Reorder the candidate pool with the arrow buttons. The gold answer is tagged so you can see exactly how the metrics respond to its position. Toggle the query type to see how abstention (NIL-first) is scored when no equivalent exists.

The interactive demo needs JavaScript. A worked example: if the gold target sits at rank 2 of 6, it contributes Hits@1 = 0, Hits@3 = Hits@5 = 1, and reciprocal rank 1/2 = 0.5.

Query: uco:Identity CLS make this a NIL query (no equivalent exists)

Where to go next

Quickstart — download the data and validate a submission in minutes.
Task 1 — Global alignment and its submission format.
Task 2 — Equivalence ranking and its submission format.
Evaluation metrics — the formal definitions.
The OAEI — the wider evaluation initiative this track belongs to.