plsimilarity
Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.5%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: AISE-TUDelft
- License: mit
- Language: Python
- Default Branch: main
- Size: 18.6 KB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 2
Metadata Files
README.md
PL Similarity
This is the replication package for the paper accepted at 2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM):
On the Impact of Language Selection for Trainingand Evaluating Programming Language Models.
By: Jonathan Katzy, Maliheh Izadi, Arie van Deursen
Setup
To calculate the similarities we need to first run the setup. In the setup we need to train the models, calculate the common tokens, and run inference for the selected tokens.
Training
We train the models according to the CodeBERTScore methodology. The code is given in Setup/train_pretrain.py for pretrained models and in Setup/train_no_pretrain.py to train the models from scratch. This uses the default script from huggingface, with the trainsteps hardcoded to 100,000, and reseting the parameters when training from scratch.
The code can be run from the command line as follows:
bash
python train_pretrain.py \
--model_name_or_path microsoft/codebert-base-mlm \
--train_file java_train \
--per_device_train_batch_size 8 \
--do_train \
--output_dir /outputs/java_pretrain
Common Tokens
After training the models, also run the Setup/tokens.py file, in order to calculate the common tokens, the results will be saved to a file.
Inference
Finally the representations of the tokens can be calculated using the Setup/eval.py code.
Similarity
To calculate the similarities, first run all the scripts form the Setup section in order to generate the required files (all intermediate outputs are saved to file), then run the functions from Similarity/similarity.py to calculate the requiered similarities.
Visualization
The visualizations for the paper are created using the scripts form the Visualization folder.
Citation
If you use this paper please cite as:
@inproceedings{katzy2023impact,
title={On the Impact of Language Selection for Trainingand Evaluating Programming Language Models},
author={Katzy, Jonathan and Izadi, Maliheh and van Deursen, Arie},
booktitle={2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM)},
pages={},
year={2023},
organization={IEEE}
}
Owner
- Name: AISE-TUDelft
- Login: AISE-TUDelft
- Kind: organization
- Repositories: 1
- Profile: https://github.com/AISE-TUDelft
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Katzy" given-names: "Jonathan" orcid: "https://orcid.org/0009-0005-9574-2414 " - family-names: "Izadi" given-names: "Maliheh" orcid: "https://orcid.org/0000-0001-5093-5523" - family-names: "van Deursen" given-names: "Arie" orcid: "https://orcid.org/0000-0003-4850-3312" title: "On the Impact of Language Selection for Training and Evaluating Programming Language Models" version: 0.0.1 doi: 10.5281/zenodo.1234 date-released: 2023-08-25 url: "https://github.com/AISE-TUDelft/PLSimilarity"
GitHub Events
Total
Last Year
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0