CAMIL: Context-Aware Multiple Instance Learning for Cancer Detection and Subtyping in Whole Slide Images (ICLR 2024 spotlight)

Tensorflow implementation for the multiple instance learning model described in the paper CAMIL (work in progress)

Pipeline Overview

Model Workflow

1. WSI Preprocessing: Segment and tile WSIs.
2. Feature Extraction: Extract 1024-dimensional feature representations using a pre-trained model.
3. Feature Transformation: A Nystromformer module enhances feature embeddings.
4. Neighbor-Constrained Attention: Captures spatial dependencies between patches.
5. Final Classification: Aggregates tile-level scores to generate the final slide classification.

Installation

Two separate conda environments are used for different stages of the workflow: - dl_torch for WSI preprocessing and feature extraction - alma for model training and evaluation

To create and activate these environments, run the following commands: bash $ conda env create --name torchenv --file torchenv.yml $ conda activate torch_env

WSI preprocessing

we follow the CLAM's WSI processing solution (https://github.com/mahmoodlab/CLAM)

bash

WSI Segmentation and Patching

python createpatchesfp.py --source DATADIRECTORY --savedir PATCHESRESULTSDIRECTORY --patchsize 256 --preset bwhbiopsy.csv --seg --patch --stitch

• source: Path to the input WSI dataset.
• save_dir: Output directory for the extracted patches.
• patch_size: Tile size (default: 256x256).
• preset: CSV preset for segmentation parameters.

Feature extraction

Extract features using pre-trained weights and save them to the output directory.

WSI Segmentation and Patching

python featureextractor/computefeats.py --weights weightdir/*.pth --dataset "PATCHESRESULTSDIRECTORY/*" --output FEATRESULTSDIRECTORY --slidedir DATA_DIRECTORY

• weight_dir: Directory containing checkpoints, one from the TCGA-NSCLC and the other from Calmeyon-16.
• output_dir: Directory where the H5 files are stored.
• slide_dir: Directory where the slides are stored.

Train

Run model training using extracted features. python run.py --experimentname EXPNAME --epoch 30 --featurepath FEATRESULTSDIRECTORY --labelfile LABELFILE --csvfile SPLITDIR --savedir WEIGHT_DIR

• experiment_name: The name of the experiment.
• epoch: The number of training epochs.
• feature_path: The path where the features are stored.
• label_file: The path to the CSV file containing labels.
• csv_file: The path to the CSV file containing data splits.

- save_dir: The directory where the weights are saved stored.

The label file should be a CSV file with the following comma-separated fields: caseid, slideid, label, slide_label

Test

Run model evaluation on test data. python run.py --experimentname EXPNAME --test --featurepath FEATRESULTSDIRECTORY --labelfile LABELFILE --csvfile SPLITDIR --savedir WEIGHT_DIR

• experiment_name: The name of the experiment.
• feature_path: The path where the features are stored.
• label_file: The path to the CSV file containing labels.
• csv_file: The path to the CSV file containing data splits.
• save_dir: The directory where the weights are saved stored.
• test: Flag indicating the test stage

If you use this code, please cite our work using:

```bibtex @inproceedings{fourkioti2024camil, title={{CAMIL}: Context-Aware Multiple Instance Learning for Cancer Detection and Subtyping in Whole Slide Images}, author={Olga Fourkioti and Matt {De Vries} and Chris Bakal}, booktitle={The Twelfth International Conference on Learning Representations}, year={2024}, url={https://openreview.net/forum?id=rzBskAEmoc} }