code-comment-analysis

Analysis of code comments using language models

Description

This repository contains code for the analysis of code comments using language models. It provides necessary scripts to generate embeddings and to perform clustering analysis on the generated embeddings.

Requirements

Dependencies

Install the Python dependencies defined in the requirements.txt. bash pip install -r requirements.txt

Accelerate

Setup accelerate: bash accelerate config

Usage

Generate embeddings

```bash usage: generate-embeddings.py [-h] [--datasetname DATASETNAME] [--datasetconfigname DATASETCONFIGNAME] [--datafiles [DATAFILES ...]] [--datasetsplit DATASETSPLIT] [--textcolumnnames [TEXTCOLUMNNAMES ...]] [--modelnameorpath MODELNAMEORPATH] [--configname CONFIGNAME] [--generationconfigfile GENERATIONCONFIGFILE] [--tokenizername TOKENIZERNAME] [--useslowtokenizer] [--outputfile OUTPUTFILE] [--seed SEED]

Do inference with a transformer model on a causal language modeling task

optional arguments: -h, --help show this help message and exit --datasetname DATASETNAME The name of the dataset to use (via the datasets library). Can also a be "csv", "json", "parquet" or "arrow" for loading a local or remote file. --datasetconfigname DATASETCONFIGNAME The configuration name of the dataset to use (via the datasets library). --datafiles [DATAFILES ...] A list of data files to use (a local dataset path or a url). --datasetsplit DATASETSPLIT The name of the split to use. Default to "train". --textcolumnnames [TEXTCOLUMNNAMES ...] The column names of the dataset to generate from. --modelnameorpath MODELNAMEORPATH Path to pretrained model or model identifier from huggingface.co/models. --configname CONFIGNAME Pretrained config name or path if not the same as modelname --generationconfigfile GENERATIONCONFIGFILE Generation config path if not the same as modelname --tokenizername TOKENIZERNAME Pretrained tokenizer name or path if not the same as modelname --useslowtokenizer If passed, will use a slow tokenizer (not backed by the 🤗 Tokenizers library). --outputfile OUTPUT_FILE The output jsonline file where the data will be saved. --seed SEED A seed for reproducible generation. ```

Example

The following example will load the data from the parquet and generate embeddings. The output will be saved to the specified jsonl file.

bash accelerate launch script/generate-embeddings.py \ --dataset_name parquet \ --data_files generated_comments_slack_256.parquet \ --text_column_names generated func_code \ --model_name_or_path andstor/gpt-j-6B-smart-contract \ --output_file embeddings_generated_comments_slack_256.jsonl \ --seed 42