Quickstart

Everything a participant runs is Python 3.12+ standard library only. Clone the distribution and run from its root.

1. Get the data

2. Task 1 — Global alignment

For each of the 6 pairs, emit one OAEI Alignment RDF (default xmlns = the alignment namespace without a trailing #; one <Cell> per = correspondence). The full spec and template are provided under tasks/global/submission-format.md.

Validate, then self-score — the references are public, with headline metrics computed using $R_{\approx}^{+}$ (with $R_{\approx}$ as secondary):

# structural check (zero-dependency); optional RelaxNG check needs libxml2-utils
python3 scripts/validate_global.py my-thinkhome-brick.rdf
xmllint --relaxng scripts/alignment.rng my-thinkhome-brick.rdf

# score one submission under the dual reference
python3 scripts/score_global.py my-submission.rdf \
    --rplus tasks/global/references/_for_use/thinkhome-brick.silver.rdf \
    --rapprox tasks/global/references/_unrepaired/thinkhome-brick.silver.unrepaired.rdf

Sanity check: a reference scored against itself gives P=R=F1=1. A MELT local-track driver is under construction (see the README).

3. Task 2 — Local equivalence ranking

For each pair, read tasks/ranking/candidates/<pair>/pools.jsonl (one JSON object per query, each with 50 candidates including the NIL IRI). Emit a JSONL submission, one line per qid, ranking that query’s candidates best-first (a permutation of the 50; rank NIL first to abstain):

{"qid": 0, "ranking": ["<best-IRI>", "...", "https://oaei.ontologymatching.org/2026/diso/NIL", "..."]}

The task is considered unsupervised; the answers (ground truths) are private, so there is no local scorer. Validate the format, then submit. We score Hits@$\{1,3,5,10\}$, MRR, and macro-average over both pairs.

python3 scripts/validate_ranking.py tasks/ranking/candidates/uco-stix/pools.jsonl    my_uco-stix.jsonl
python3 scripts/validate_ranking.py tasks/ranking/candidates/stix-d3fend/pools.jsonl my_stix-d3fend.jsonl

Full spec + worked example: tasks/ranking/submission-format.md.

4. Submit

The evaluation window runs from 19 July to 1 September 2026, 00:00 Anywhere on Earth (AoE) (provisional). The submission channel will be announced on the track page; until then, contact the track organisers (see README). Organisers verify, reproduce where possible, and publish results to the baseline leaderboard.