PhenoMIL

Official PyTorch implementation of Learning from Limited Phenotype-Level Annotations for Promoting Multiple Instance Learning in Endoscopic Helicobacter pylori Infection Diagnosis.

PhenoMIL targets endoscopic Helicobacter pylori infection diagnosis under mixed supervision, combining limited phenotype-level annotation with patient-level bag labels. It first learns transferable, phenotype-aware image representations through FPS-SL pretraining and then transfers the encoder to CIMIL for clinically informed patient-level multiple instance learning. The overall training pipeline includes:

1. Warm-up: initialize the instance and bag modules.
2. FPS-SL: fine-grained phenotype-level semi-supervised pretraining at the image level.
3. CIMIL: clinically informed multiple instance learning for patient-level diagnosis.

Repository layout

PhenoMIL/
├── 1_FPS-SL/                 # Stage 1: phenotype-level semi-supervised pretraining
│   ├── train.py
│   ├── configs/
│   ├── lib/
│   └── utils/
├── 2_CIMIL/                  # Stage 2: bag-level diagnosis
│   ├── train_MIL.py
│   ├── test_MIL_comparison.py
│   ├── configs/
│   ├── lib/
│   └── utils/
├── figs/
│   ├── figure1.pdf
│   ├── figure1.png
│   ├── figure2_1_v3.pdf
│   └── figure2_1_v3.png
├── requirements.txt
└── README.md

Environment

We recommend Python 3.9+ and CUDA-enabled PyTorch.

conda create -n phenomil python=3.9 -y
conda activate phenomil
pip install -r requirements.txt

Data preparation

The repository expects local image folders plus text index files.

1. Bag-level index files

The example index format is provided in:

• 1_FPS-SL/configs/multi_center_training/example.txt
• 2_CIMIL/configs/multi_center_training/example.txt

Each line should contain a bag folder path and its binary diagnosis label:

/path/to/patient_bag_001    1
/path/to/patient_bag_002    0
...

For some downstream utilities, an optional third column may be used for additional document labels.

2. Phenotype annotation JSON

FPS-SL and part of the CIMIL dataloading pipeline require an image-level phenotype annotation JSON file.

Set it through an environment variable before training:

export PHENOMIL_ANNOTATION_JSON=/path/to/phenotype_annotations.json

If this variable is not set, the code will fall back to the original internal path only when that path exists locally.

3. Pretrained backbone weights

By default, FPS-SL initializes the PVT backbone with ImageNet-pretrained pvt_v2_b2 weights through timm.

Override the default only if you want to initialize from a custom checkpoint:

--pretrained_path /path/to/custom_checkpoint.pth

Stage 1: FPS-SL pretraining

Run FPS-SL from the 1_FPS-SL directory.

1) Warmup

The warmup stage contains an instance phase followed by a bag phase.

Instance phase

cd 1_FPS-SL
CUDA_VISIBLE_DEVICES=0,1 python -u train.py \
  --dist-url tcp://localhost:10005 \
  --multiprocessing-distributed \
  --world-size 1 \
  --rank 0 \
  --seed 123 \
  --dataset endoscopy \
  --arch pvt_v2_b2 \
  --pretrained \
  --epochs 100 \
  --batch-size 256 \
  --exp-dir snapshots/instance_warmup \
  --train_file ./configs/multi_center_training/train_data.txt \
  --train_bag_file ./configs/multi_center_training/train_data.txt \
  --test_file ./configs/multi_center_training/eval_data.txt \
  --add_bag_head \
  --prot_start 10000 \
  --sup_bag_start 0 \
  --temperature 0.07 \
  --cosine \
  --train_only_labeled \
  --train_no_neither \
  --train_constrain_no_neither_nums 500

Bag phase

cd 2_CIMIL
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 --master_port 29502 train_MIL.py \
  --config configs/CIMIL.yaml \
  --checkpoint_dir snapshots/fps_sl_warmup \
  --finetune fps_sl \
  --finetune_path /path/to/instance_warmup_checkpoint.pth.tar \
  --pooling Mean \
  --frozen True

2) Full FPS-SL training

The full Stage-1 run can then continue from the instance-phase checkpoint:

cd 1_FPS-SL
CUDA_VISIBLE_DEVICES=0,1 python -u train.py \
  --dist-url tcp://localhost:10005 \
  --multiprocessing-distributed \
  --world-size 1 \
  --rank 0 \
  --seed 123 \
  --dataset endoscopy \
  --arch pvt_v2_b2 \
  --epochs 400 \
  --semi_start 10 \
  --batch-size 256 \
  --exp-dir snapshots/fps_sl \
  --resume /path/to/fps_sl_warmup_checkpoint.pth.tar \
  --train_file ./configs/multi_center_training/train_data.txt \
  --train_bag_file ./configs/multi_center_training/train_data.txt \
  --test_file ./configs/multi_center_training/eval_data.txt \
  --temperature 0.07 \
  --cosine \
  --stable_queue \
  --train_no_neither \
  --train_constrain_no_neither_nums 500 \
  --pseudo_bag_sup \
  --proto_cont \
  --proto_scoring v2

Notes:

• The default augmentation file is configs/transform.yaml.
• Important outputs are written under --exp-dir, including checkpoints and logs.

Stage 2: CIMIL training

Run CIMIL from the 2_CIMIL directory:

cd 2_CIMIL
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 --master_port 29502 train_MIL.py \
  --config configs/CIMIL.yaml \
  --checkpoint_dir snapshots/cimil \
  --finetune fps_sl \
  --finetune_path /path/to/fps_sl_checkpoint.pth.tar \
  --pooling CIMIL \
  --frozen True

Key options:

• --pooling: downstream MIL backbone / aggregator.
• --finetune_path: FPS-SL checkpoint used to initialize the encoder.
• --finetune: use fps_sl for Stage-1 checkpoints.
• --pooling: use CIMIL for the full PhenoMIL aggregator.
• --checkpoint_dir: output directory for logs and checkpoints.

Evaluation

The main comparison/evaluation script is:

cd 2_CIMIL
python test_MIL_comparison.py \
  --resume /path/to/cimil_checkpoint.pth \
  --index_root configs/multi_center_training/eval_data.txt \
  --output_txt results/eval_summary.txt

This script was originally used for our internal multicenter evaluation workflow. You may still need to adapt center names in the reporting logic if your test cohort structure differs from ours.