tlcs
The repository for the replication package of the paper "On the Effectiveness of Transfer Learning for Code Search."
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: zenodo.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.5%) to scientific vocabulary
Repository
The repository for the replication package of the paper "On the Effectiveness of Transfer Learning for Code Search."
Basic Info
- Host: GitHub
- Owner: pasqualesalza
- License: cc-by-4.0
- Default Branch: main
- Homepage: https://tl-codesearch.netlify.app
- Size: 7.05 GB
Statistics
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 1
- Releases: 1
Metadata Files
README.md
On the Effectiveness of Transfer Learning for Code Search
This repository represents the replication package for the paper:
On the Effectiveness of Transfer Learning for Code Search
The Transformer architecture and transfer learning have marked a quantum leap in natural language processing, improving the state of the art across a range of text-based tasks. This paper examines how these advancements can be applied to and improve code search. To this end, we pre-train a BERT-based model on combinations of natural language and source code data and fine-tune it on pairs of StackOverflow question titles and code answers. Our results show that the pre-trained models consistently outperform the models that were not pre-trained. In cases where the model was pre-trained on natural language "and" source code data, it also outperforms an information retrieval baseline based on Lucene. Also, we demonstrated that the combined use of an information retrieval-based approach followed by a Transformer leads to the best results overall, especially when searching into a large search pool. Transfer learning is particularly effective when much pre-training data is available and fine-tuning data is limited. We demonstrate that natural language processing models based on the Transformer architecture can be directly applied to source code analysis tasks, such as code search. With the development of Transformer models designed more specifically for dealing with source code data, we believe the results of source code analysis tasks can be further improved.
The paper is published in the journal IEEE Transactions on Software Engineering (TSE).
In our work, we design a Multimodal Embedding Model (MEM) architecture, which leverages the predictive capabilities of BERT, a state-of-the-art, Transformer-based NLP model, and makes use of transfer learning to improve the performance on code search.
In this replication package, we provide all the data and scripts we used in our study.
:openfilefolder: Organization
The repository is organized as follows:
data/contains the data of the replication packagesource/the source code for the pre-training and fine-tuning, and mining of StackOverflow datamodels/the pre-trained BERT models, ready to be loaded and used for other tasks, or to replicate the rest of our studysodata/the StackOverflow data description and files we collected to fine-tune our MEM architectureresults/the results data we collectedanalysis/the full analysis we performed, in form of a Python Jupyter Notebook
docs/contains the source code for the website where we documented our experiments
:books: How to cite this dataset
If you would like to cite the dataset, please use the following BibTeX snippet:
bibtex
@article{salza_transfer_2022,
author = {Salza, Pasquale and Schwizer, Christoph and Gu, Jian and Gall, Harald C.},
title = {{On the Effectiveness of Transfer Learning for Code Search}},
journal = {IEEE Transactions on Software Engineering (TSE)},
year = {2022},
}
:balance_scale: License
This replication package is licensed under the terms of the Creative Commons Attribution 4.0 International License. Please see the LICENSE file for full details.
:pray: Credits
- Pasquale Salza - University of Zurich, Switzerland
- Christoph Schwizer - University of Zurich, Switzerland
- Jian Gu - University of Zurich, Switzerland
- Harald C. Gall - University of Zurich, Switzerland
Owner
- Name: Pasquale Salza
- Login: pasqualesalza
- Kind: user
- Location: Zurich, Switzerland
- Company: kaiko.ai
- Website: https://pasqualesalza.com
- Repositories: 3
- Profile: https://github.com/pasqualesalza
Hello! I'm Pasquale, an Italian software engineer who loves geek stuff.
GitHub Events
Total
Last Year
Committers
Last synced: over 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| Pasquale Salza | p****a@g****m | 11 |
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 1
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 1
- Total pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 1
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 1
- Pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- xiangzheng666 (1)