Track: Track1; Team name: SweetLesson; Model: HOD-GNN

[AaravG42]

HOD-GNN — first public implementation of "On the Expressive Power of GNN Derivatives" (ICLR 2026)

Paper: Eitan, Eliasof, Gelberg, Frasca, Bar-Shalom, Maron — arXiv:2510.02565 / OpenReview Tvat33IDmK.
To our knowledge no public implementation exists (checked the authors' pages, GitHub code search, and the paper's citations as of 2026-07-28); this backbone was written from the paper alone.

Why this model for this challenge

Message passing is bounded by 1-WL and provably cannot count triangles (Chen et al., NeurIPS 2020). HOD-GNN's motivating example is exactly the triangle-counting task: for a base GNN computing A³X, the derivative of node v's output w.r.t. its own input is (A³)_vv — six times the triangles at v. HOD-GNN turns the derivatives of a base MPNN into learnable, structure-aware node features that restore what 1-WL is blind to, with expressivity between encoding-augmented MPNNs and Subgraph GNNs (paper §4). It is also a direct conceptual extension of our TopoPolynormer submission (#353): Theorem 4.2 of the paper proves the derivative diagonal of a suitably initialised base MPNN equals the random-walk structural encoding — so the hand-crafted RWSE we injected there is here learned end-to-end, strictly more expressively.

What is implemented

The paper's own empirical architecture (its Appendix F — used for every reported experiment):

• 1-HOD-GNN with first-order derivatives: a deep-and-narrow base GIN (width 8, depth 6 here) whose exact input-Jacobian is propagated analytically alongside the features (the message-passing derivative algorithm of the paper's §3.2.1 / Appendix D; for first order, Faà di Bruno's formula reduces to the chain rule). The propagation uses the same sparse operator as feature aggregation and is fully differentiable, so the base GIN is trained through the derivative computation — the paper's key algorithmic contribution.
• Factorial residual (Eq. 73): per-layer features and per-layer diagonal Jacobians concatenated with 1/t! scaling (keeping every depth's diagonal matches the RWSE correspondence, which needs all walk lengths).
• Pointwise encoder U^node on the diagonal (Eq. 74), then a downstream GIN.
• Deliberately omitted, matching the paper's experiments: output-level derivatives D^out, k≥2 mixed derivatives, IGN encoders.
• Batch-aware by construction: the Jacobian source axis is indexed locally per graph; a graph's outputs (including all derivative channels) are bit-identical alone vs. inside a batch (tested).

Documented deviations and an insight for reusers

1. Default aggregation is the row-normalised (random-walk) operator from the constructive proof of Theorem 4.2 — a member of the linear-aggregation family the derivative algorithm supports (Eq. 28) — rather than GIN sum. Reason, discovered empirically and reproducible: the challenge harness fixes a single Adam lr=1e-3, while the paper uses a separate base-MPNN lr of 1e-4 with warmup+cosine on small molecular graphs. With sum aggregation, the paper's Appendix-F initialisation produces walk-count-scale derivative features ((A^t)_vv) that explode on the denser GraphUniverse regimes (initial loss ~2×10⁷, oscillatory training that patience-10 early stopping freezes at a bad checkpoint). The mean operator keeps the derivative diagonals in [0,1] — at initialisation they are exactly the k-step RW return probabilities — and trains stably. Both variants are implemented and both were run on the full grid (table below).
2. Derivatives are taken w.r.t. the projected (64→8) input features; the paper's "initial features X" are likewise an embedding of the raw input.

Tests (100% line coverage of the backbone)

• Analytic vs autograd: the hand-propagated per-layer diagonal Jacobians match torch.autograd.functional.jacobian exactly (atol 1e-10, float64), for every activation × aggregation combination, inside a batch.
• Theorem-4.2 equivalence: under the paper's initialisation, derivative diagonals equal dense-matrix-power RW return probabilities (mean) / closed-walk counts (sum) exactly.
• Beyond-1-WL fixture: the derivative features separate C₆ from 2×C₃ (identical 1-WL colourings).
• Batch isolation (batched == individual, diff 0), permutation equivariance, gradient flow through the derivative computation into the base weights, dropout/validation branches.

Results — full 72-run GraphUniverse grid + OOD evaluation

Generated with run_challenge_grid() + save_challenge_artifacts() from 2026_tdl_challenge/utils.py (unmodified). Note: the evaluation notebook's integrity-hash cell currently rejects the unmodified upstream notebook (stored hash mismatch), same as noted in #345/#353, so the backend was called directly — identical pipeline. Committed results.json is the default config (study 2026-07-27_22-10-00-rw).

Config	CD accuracy (in-dist / OOD ↑)	Triangle MSE/total-tri (in-dist / OOD ↓)
HOD-GNN default (mean, RWSE init)	0.439 / 0.358	0.080 / 2.40
HOD-GNN ablation (sum, standard init)	0.432 / 0.355	0.174 / 10.6
Polynormer baseline (#345)	0.455 / 0.350	0.872 / 35.1

• ~11× (in-dist) / ~15× (OOD) lower triangle-counting error than our faithful Polynormer baseline — direct empirical confirmation of the derivative mechanism restoring sub-structure counting.
• Best OOD community-detection accuracy among our submissions (0.358) — consistent with the degree-normalised derivative features being invariant across the density/degree grid (~92% of all evaluations are OOD).
• Backbone parameters: 62,353 (Polynormer baseline: 76,160). Worst-case grid setting: 160 ms/iteration, 7.3 GiB peak (batch of 16 × 300-node graphs).

Files

• topobench/nn/backbones/graph/hod_gnn.py — backbone (module docstring contains the full method description, faithfulness notes, and references)
• configs/model/graph/hod_gnn.yaml — model config (both tasks)
• test/nn/backbones/graph/test_hod_gnn.py — test suite
• test/pipeline/test_pipeline.py — pipeline registration
• 2026_tdl_challenge/outputs/2026-07-27_22-10-00-rw/results.json — grid results (default config)

Team SweetLesson · Track 1 (GNNs) · companion submissions: #345 (Polynormer, faithful baseline), #353 (TopoPolynormer, hand-crafted RWSE) — together the three PRs progress from a faithful transformer baseline (#345), to a hand-crafted structural encoding (#353), to this PR, where that encoding is generalised and learned end-to-end.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB