Skip to content

Any backbone for linear segmentation#777

Open
liopeer wants to merge 7 commits into
mainfrom
lionel-any-backbone-linseg
Open

Any backbone for linear segmentation#777
liopeer wants to merge 7 commits into
mainfrom
lionel-any-backbone-linseg

Conversation

@liopeer

@liopeer liopeer commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

What has changed and why?

This is the beginning of a refactoring effort

  • adds a new way for configs
  • allows easy extension of linear segmentation for more backbones

Follow-ups will rename things away from DINOv2 and to generic names. After that I will move on to object detection, but don't want this to interfere with #772 etc. (ECViT implementations).

How has it been tested?

  • I'll add unit tests later during clean-up (there are anyway renamings still missing etc.)

Did you update CHANGELOG.md?

  • Yes
  • Not needed (internal change)

Did you update the documentation?

  • Yes
  • Not needed (internal change without effects for user)

Copilot AI review requested due to automatic review settings June 12, 2026 14:57
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new “linear semantic segmentation” task-model implementation that is intended to be extendable across multiple backbone families by adding a small model-name → backbone/config registry, and it refactors internal model-wrapper typing to distinguish multi-scale ViT vs CNN capabilities.

Changes:

  • Added a new linear semantic segmentation task model (LinearSemanticSegmentation) with tiling/untile inference utilities and a dedicated training loop.
  • Introduced a registry-based configuration layer for mapping *-linear model names to a backbone + default backbone args.
  • Refined model wrapper / package typing to express “multi-scale feature” capability for ViTs vs CNNs, and updated DINOv2/DINOv3 wrappers accordingly.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 21 comments.

Show a summary per file
File Description
src/lightly_train/_task_models/linear_semantic_segmentation/transforms.py Adds default train/val transform arg presets for linear semantic segmentation.
src/lightly_train/_task_models/linear_semantic_segmentation/train_model.py Adds the training/validation loop for linear semantic segmentation.
src/lightly_train/_task_models/linear_semantic_segmentation/task_model.py Adds the inference-time task model (tiling, postprocess) and backbone wiring.
src/lightly_train/_task_models/linear_semantic_segmentation/config.py Adds registry entries mapping *-linear model names to backbone definitions.
src/lightly_train/_models/package.py Introduces MultiScaleFeaturePackage and a model-name parsing protocol.
src/lightly_train/_models/model_wrapper.py Splits multi-scale feature protocols into ViT/CNN variants and adds stride/patch-size capability.
src/lightly_train/_models/dinov3/dinov3_vit.py Updates DINOv3 ViT wrapper to provide patch size and new protocol typing.
src/lightly_train/_models/dinov3/dinov3_convnext.py Updates DINOv3 ConvNeXt wrapper typing and adds multi-scale stride reporting.
src/lightly_train/_models/dinov3/dinov3_src/models/convnext.py Adds a type annotation for downsample_layers.
src/lightly_train/_models/dinov3/dinov3_package.py Marks DINOv3 as a multi-scale-feature package and adds protocol typing.
src/lightly_train/_models/dinov2_vit/dinov2_vit.py Updates DINOv2 ViT wrapper to provide patch size and new protocol typing.
src/lightly_train/_models/dinov2_vit/dinov2_vit_package.py Marks DINOv2 as a multi-scale-feature package and adds protocol typing.
src/lightly_train/_configs/model_registry.py Adds a generic registry utility used for model/config lookup.
src/lightly_train/_configs/config.py Adds ConfigsNamespace to enforce “namespace-only” config containers.
src/lightly_train/_commands/train_task_helpers.py Switches the training task helper to import the new linear semantic segmentation train model.

Comment thread src/lightly_train/_task_models/linear_semantic_segmentation/transforms.py Outdated
Comment thread src/lightly_train/_task_models/linear_semantic_segmentation/transforms.py Outdated
Comment thread src/lightly_train/_task_models/dinov2_linear_semantic_segmentation/config.py Outdated
Comment thread src/lightly_train/_task_models/linear_semantic_segmentation/task_model.py Outdated
Comment thread src/lightly_train/_task_models/linear_semantic_segmentation/task_model.py Outdated
Comment thread src/lightly_train/_models/dinov3/dinov3_vit.py
Comment thread src/lightly_train/_models/dinov2_vit/dinov2_vit.py
Comment thread src/lightly_train/_models/dinov2_vit/dinov2_vit.py
Comment thread src/lightly_train/_models/dinov3/dinov3_convnext.py
Comment thread src/lightly_train/_models/dinov3/dinov3_convnext.py
Comment thread src/lightly_train/_models/dinov2_vit/dinov2_vit_package.py
Comment thread src/lightly_train/_models/dinov3/dinov3_package.py
Comment thread src/lightly_train/_models/dinov3/dinov3_vit.py
...


class MultiScaleFeaturePackage(Package):

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Eventually all packages should implement this.

Comment thread src/lightly_train/_task_models/dinov2_linear_semantic_segmentation/config.py Outdated
DinoVisionTransformer,
from lightly_train._models.model_wrapper import ModelWrapper
from lightly_train._models.package import MultiScaleFeaturePackage
from lightly_train._task_models.dinov2_linear_semantic_segmentation.config import (

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do a lot of renaming from dinov2_linear_semantic_... to just semantic_ in a follow up.

@liopeer

liopeer commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

/review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants