CNA-Modules

Official code for the paper Graph Neural Networks Need Cluster-Normalize-Activate Modules accepted at NeurIPS 2024.

Installation

Dockerfile

We provide a Dockerfile for ease of reproducibility. See the Docker Docs on How to get started.

Via installation script

Navigate into the directory cna_modules. Then execute the following command to make the script executable: bash chmod +x install_script.sh

After that, execute the script via: bash bash -i install_script.sh

When the script is executed successfully, activate the conda environment via: bash conda activate cluster-normalize-activate

Via conda environment

To install the necessary libraries, you can create a conda environment using the following command:

conda env create -f environment.yml

The required libraries are saved in the environment.yml file. After the installation is complete, activate the environment with the following command:

conda activate cna-modules

Alternatively, you can run the following command to install:

make install

Project Structure

The Python scripts can be found in the src directory, and the figures are located in the images directory.

Usage

To use the project, navigate to the ~/cna_modules/src directory and run the appropriate script.

python scripts/execute_experiments.py

To adapt the parameters, open the file model_params.py in the utils directory, and you can here see the possible options to choose or adapt:

``` experimentnumber = ... # number of experiment epochs = [...] # number of epochs (list) modeltype = ... # to define the model type numhiddenfeatures = [...] # number of hidden features (list) lrmodel = [...] # learning rate for the model (list) lractivation = [...] # learning rate for the activations (list) weightdecay = [...] # weight decay for both (list) clusters = [...] # number of clusters (list) numlayers = [...] # number of layers (list) numactivation = [...] # number of activations inside RPM (list) n = ... # numerator m = ... # denominator reclusteroption = ... activationtype = [...] # activation type (list) mode = [...] # distance metric type (list) withclusters = [...] # flag for clustering (list) usecoefficients = ... # flag for use of coefficients in our Rationals device = torch.device("cuda" if torch.cuda.isavailable() else "cpu")

Types: Planetoid, CitationFull, Amazon, WikipediaNetwork, WebKB

name of the dataset (Cora, CiteSeer, PubMed, Cora_ML, chameleon, Photo etc.)

setdataset = ... # here to set the dataset tasktype = ... # here to set the task type ```

But you can also use the predefined configurations as listed in the directory utlis/configs. To execute an experiment this way you can run: python scripts/execute_experiments.py --config [name of configuration] --num_seeds [num of seeds] An excerpt from all accessible configurations:

CiteSeer (Node Classification): text citeseer_4_gatconv citeseer_2_gcnconv citeseer_4_gcnconv citeseer_8_gcnconv citeseer_16_gcnconv citeseer_32_gcnconv citeseer_64_gcnconv Cora (Node Classification): text cora_4_sageconv cora_2_gcnconv cora_4_gcnconv cora_8_gcnconv cora_16_gcnconv cora_32_gcnconv cora_64_gcnconv cora_96_gcnconv corafull_2_transformerconv Others (Node Classification): text squirrel_2_dirgcnconv computers_2_transformerconv chameleon_2_dirgcnconv texas_2_sageconv wisconsin_2_transformerconv dblp_4_transformerconv photo_4_transformerconv pubmed_2_transformerconv Ogbn-arxiv (Node Property Prediction): text ogbn-arxiv_4_nodeproppred_sageconv ogbn-arxiv_4_nodeproppred_gcnconv Others (Node Regression): text chameleon_2_node_regression_transformerconv squirrel_2_node_regression_transformerconv We ask you kindly to have a look at src/utils/configs/ to explore other options.

Contributors

• Arseny Skryagin, Felix Divo, Amin Ali

How to cite

latex @inproceedings{Skryagin_Graph_Neural_Networks_2024, author = {Skryagin, Arseny and Divo, Felix and Ali, Mohammad Amin and Dhami, Devendra Singh and Kersting, Kristian}, month = dec, series = {The Thirty-eighth Annual Conference on Neural Information Processing Systems}, title = {{Graph Neural Networks Need Cluster-Normalize-Activate Modules}}, url = {https://openreview.net/forum?id=faj2EBhdHC}, year = {2024} }

License

This project is licensed under the MIT License - see the LICENSE file for details.