https://github.com/artificialzeng/gct-pytorch
Graph Convolutional Transformer implemented in pytorch
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.1%) to scientific vocabulary
Last synced: 9 months ago
·
JSON representation
Repository
Graph Convolutional Transformer implemented in pytorch
Basic Info
- Host: GitHub
- Owner: ArtificialZeng
- Default Branch: main
- Size: 20.5 KB
Statistics
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of dchang56/gct-pytorch
Created about 2 years ago
· Last pushed over 5 years ago
https://github.com/ArtificialZeng/gct-pytorch/blob/main/
# Graph Convolutional Transformer in Pytorch
I've reimplemented the original tensorflow implementation of [this paper](https://arxiv.org/pdf/1906.04716.pdf) by Choi et al, following the materials provided.
The step-by-step instructions are pretty much the same as the original repo: https://github.com/Google-Health/records-research/tree/master/graph-convolutional-transformer
I did not implement the synthetic parts of the paper, and I've left quite a bit of alternative model architecture and training details in the code (which I think is appropriate given the engineering-heavy and experimental nature of the project).
Here is an example of a bash script to run experiments. There are two possible tasks: mortality prediction and readmission prediction. Just assign either 'expired' or 'readmission' as the `LABEL_KEY`
```bash
export DATA_DIR='data dir'
export CUDA_VISIBLE_DEVICES="2"
LABEL_KEY=readmission
for LR in 1e-3 1.5e-3 2e-3; do
for DROPOUT in 0.4 0.5 0.6 0.7; do
OUTPUT_DIR='output dirs_${LR}_${DROPOUT}'
mkdir -p $OUTPUT_DIR
python train.py \
--data_dir $DATA_DIR \
--fold 50 \
--output_dir $OUTPUT_DIR \
--use_prior \
--use_guide \
--output_hidden_states \
--output_attentions \
--do_train \
--do_eval \
--do_test \
--label_key $LABEL_KEY \
--max_steps 1000000 \
--hidden_dropout_prob $DROPOUT \
--num_stacks 2 \
--learning_rate $LR
done
done
done
```
## Comments
In my experience, training is quite unstable, and replicating the results exactly as presented in the paper has been a challenge. Still, this reimplementation was a decent learning experience.
Owner
- Name: Dr. Artificial曾小健
- Login: ArtificialZeng
- Kind: user
- Location: Beijing
- Website: https://blog.csdn.net/sinat_37574187?type=blog
- Repositories: 171
- Profile: https://github.com/ArtificialZeng
LLM practitioner/engineer, AI/ML/DL Quant