This repository is the reproducibility record for a large-scale study of English Word Sense Disambiguation (WSD): relabel all 226,036 SemCor instances with current language models, retrain established systems under controlled conditions, and use the resulting supervision to train Glite LENS, a 298M-parameter ModernBERT bi-encoder.
The public relabeled-data repository is the simplest entry point for using the datasets. It provides aligned WordNet 3.0 label layers and per-instance provenance for two full SemCor relabeling runs:
| Variant | Instances | Model labels | Fallback or unresolved | Disagreement with SemCor gold |
|---|---|---|---|---|
GPT-5.5 low, lexEn p003 |
226,036 | 226,032 | 4 | 21.30% |
Gemma 4 31B reasoning, lexEn p003 |
226,036 | 224,905 | 1,131 | 21.85% |
Disagreement is not an error rate. It identifies labels on which an LLM and the SemCor annotation differ; the controlled retraining experiments test whether those changes improve downstream WSD.
The complete research records are available here:
- GPT-5.5 dataset card
- Gemma dataset card
- Recovered zero-fallback Gemma dataset card
- GPT-5.5 drop-in dataset card
Glite LENS is a two-tower gloss bi-encoder with independent
answerdotai/ModernBERT-base context and
gloss encoders. It uses target-subword context pooling, structured WordNet glosses, dot-product
scoring, and in-batch gloss-union negatives. The canonical study contains three relabeled-data
seeds, an original-label control, and a CLS-pooling ablation.
Micro-F1 on the canonical evaluation surfaces:
| Training labels | Raganato ALL | Maru 2022 ALL_NEW | lexEn v1 |
|---|---|---|---|
| Original SemCor, seed 42 | 80.84 | 82.14 | 83.52 |
| GPT-5.5-relabeled SemCor, three-seed mean | 83.64 | 87.37 | 90.48 |
| Label-axis gain | +2.80 | +5.23 | +6.96 |
The serving benchmark measured $0.126 per million items at steady state on one RTX 5090 with the gloss gallery precomputed, batched bf16 inference, and model-load time excluded.
Canonical LENS materials:
- Model card and checkpoint inventory
- Prediction-set card
- Canonical numbers
- Detailed results
- Training and evaluation code
The checkpoint files are managed with Git LFS. Operational logs were removed before public release; the retained configurations, split manifests, selected-checkpoint records, predictions, metrics, and hashes provide the reproducibility trail.
The project first reproduced BEM, ESCHER, and ConSeC on the shared evaluation surfaces. It then generated the two SemCor label layers, retrained each established architecture without changing its central recipe, compared the resulting predictions, and trained the canonical LENS grid. The results dashboard aggregates all completed tasks and registered artifacts.
Useful waypoints:
| Stage | Tasks |
|---|---|
| Source datasets and shared loaders | t0007, t0008, t0009, t0010 |
| Baseline reproductions | t0011 BEM, t0012 ESCHER, t0013 ConSeC |
| Full-corpus relabeling | t0014 GPT-5.5, t0015 Gemma |
| Controlled retraining | t0016–t0021 |
| Cross-system analysis | t0022 |
| Glite LENS | t0032, t0034 |
The work was carried out with the
Glite Autonomous Research Framework, a file-based,
verifier-driven workflow for research with coding agents. Each tasks/tXXXX_* directory is a
self-contained research unit with its question, plan, code, registered assets, results, and
machine-checkable status. The framework and its design are described in
Glite ARF: Verifier-Driven Research with Parallel LLM Coding Agents.
The accompanying WSD manuscript is English Word Sense Disambiguation in 2026: When the Labels
Become the Bottleneck. Its arXiv identifier will be added after assignment; CITATION.cff already
contains the paper's author and title metadata.
Size warning: this repository tracks 164 Git LFS objects totalling about 52 GB — model
checkpoints, split checkpoint parts, drop-in corpora, and prediction sets. A default git clone
followed by git lfs pull downloads all of it. Clone with LFS download disabled and fetch only the
artifacts you need, or use the direct download links below
without cloning at all.
Clone without downloading LFS content and install the locked Python environment:
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/GliteTech/research-semcor-relabeling.git
cd research-semcor-relabeling
uv sync --frozenFetch individual large artifacts on demand, for example:
git lfs pull --include="tasks/t0034_glite_lens_canonical/assets/model/**"
git lfs pull --include="tasks/t0016_retrain_bem_gpt55/assets/model/**"Run the repository checks with:
uv run python doctor.py
uv run pytestThis is a research archive rather than a single installable WSD package. For dataset consumption, start with the public relabeled-data repository. For LENS reproduction, use the task-local code and saved configuration documented in the model card.
Any file in the repository can be downloaded individually through GitHub's LFS media endpoint:
https://media.githubusercontent.com/media/GliteTech/research-semcor-relabeling/refs/heads/main/<path-in-repo>
For the relabeled SemCor label layers, prefer the small, LFS-free llm-relabeled-semcor repository. The key artifacts hosted in this repository are listed below. For multi-part checkpoints, download every part; each model card documents part order, reassembly, and SHA-256 hashes.
| Artifact | Size | Download |
|---|---|---|
| Glite LENS, GPT-5.5 labels, seed 42 (canonical) | 1.2 GB | best_model.ckpt |
| Glite LENS, GPT-5.5 labels, seed 43 | 1.2 GB | best_model.ckpt |
| Glite LENS, GPT-5.5 labels, seed 44 | 1.2 GB | best_model.ckpt |
| Glite LENS, original-label control | 1.2 GB | best_model.ckpt |
| Glite LENS, CLS-pooling ablation | 1.2 GB | best_model.ckpt |
| BEM baseline (original SemCor) | 876 MB | best_model.ckpt |
| BEM retrained on SemCor-GPT5.5 | 876 MB | best_model.ckpt |
| BEM retrained on SemCor-Gemma | 876 MB | best_model.ckpt |
| ESCHER baseline | 4.9 GB | part00, part01 |
| ESCHER retrained on SemCor-GPT5.5 | 4.9 GB | part00, part01 |
| ESCHER retrained on SemCor-Gemma | 4.9 GB | part00, part01 |
| ConSeC baseline | 4.9 GB | part-00, part-01 |
| ConSeC retrained on SemCor-GPT5.5 | 5.0 GB | part-00, part-01, part-02, part-03, part-04 |
| ConSeC retrained on SemCor-Gemma | 4.8 GB | part00, part01 |
| SemCor-GPT5.5 drop-in training corpus | 53 MB | semcor.data.xml, semcor.gold.key.txt, zip |
| SemCor-Gemma complete drop-in training corpus | 46 MB | semcor.data.xml, semcor.gold.key.txt |
Per-instance prediction files are small (under 11 MB each) and live under
tasks/*/assets/predictions/; fetch any of them with the same URL pattern, for example the
canonical LENS predictions on Raganato ALL:
predictions-raganato-all.jsonl.gz
tasks/ Research tasks, code, registered assets, predictions, and results
overview/ Generated project-wide results and artifact dashboard
arf/ Glite ARF framework snapshot used to run and verify the study
meta/ Asset, metric, category, and task-type specifications
project/ Project definition and budget metadata
This is a mixed-license research repository. The root Apache License 2.0 covers original code and documentation authored by Glite Tech Ltd. It does not override licenses attached to third-party software, models, datasets, corpora, or papers.
- Glite LENS checkpoint weights: CC BY-NC 4.0; see the model license.
- SemCor source data and Glite's relabeled SemCor datasets: WordNet 1.6 / Semantic Concordance 1.6 license; see the Princeton license, the SemCor 3.0 mapping notice, and the copies accompanying each dataset release.
- BEM-derived material: CC BY-NC 4.0.
- ESCHER- and ConSeC-derived material: CC BY-NC-SA 4.0.
- Other third-party materials retain their respective terms.
See Third-Party Notices, the LICENSES/ directory, and the
paper PDF rights register for sources, attribution, and scope.
Please use the metadata in CITATION.cff. The preferred citation is the
accompanying paper:
Vassili Philippov, Amro Salman, Dmitrii Andreev, Penny Hands, Emil Kaiumov, Pavel Katunin, and Anton Nikolaev. “English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck.” 2026.