guardrails
VSCode extension to help developers set up guardrails around their functions, by helping them disambiguate purpose statements.
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 1 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org, acm.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (14.6%) to scientific vocabulary
Keywords
Repository
VSCode extension to help developers set up guardrails around their functions, by helping them disambiguate purpose statements.
Basic Info
- Host: GitHub
- Owner: mrigankpawagi
- License: mit
- Language: Python
- Default Branch: master
- Homepage: https://marketplace.visualstudio.com/items?itemName=MrigankPawagi.guardrails
- Size: 11.2 MB
Statistics
- Stars: 5
- Watchers: 1
- Forks: 1
- Open Issues: 2
- Releases: 0
Topics
Metadata Files
README.md
GuardRails: Automated Suggestions for Clarifying Ambiguous Purpose Statements
VSCode extension to help developers write complete and unambiguous docstrings for their Python functions. Install it now from the Visual Studio Code Marketplace.
[!IMPORTANT] Our paper was presented at the 6th Annual COMPUTE Conference by ACM India. Check it out here (or here on arXiv).
[!WARNING]
Due to recent updates in the Copilot's VSCode extension, functionality critical for GuardRails is no longer supported. Please hang on while we update GuardRails. If you are interested in contributing, check out this.
Requirements
You need Python 3 installed on your system to use GuardRails.
Note: GuardRails is fully tested only on Windows and currently provides experimental support for Linux.
Installing Python Dependencies
bash
$ python -m pip install hypothesis hypothesis[cli] hypothesis[ghostwriter] black
$ python -m pip install git+https://github.com/mrigankpawagi/mutpy.git#egg=mutpy
$ python -m pip install func-timeout
A Motivating Example
```python def first_nonzero(nums: list[float]) -> float: """ Return the first non-zero value in nums.
>>> first_nonzero([0.0 , 3.7 , 0.0])
3.7
"""
```
This is an example of an incomplete function definition in Python. It has a purpose statement (called the docstring) which contains a functional example (called a doctest). The purpose statement is ambiguous because it does not specify the intended behavior on lists with no non-zero elements.
Contributions
Heuristic
We propose a novel heuristic that suggests such inputs using Large Language Models (LLMs). Using these suggestions, the programmer may choose to clarify the purpose statement (e.g., by providing a functional example that specifies the intended behavior on such an input). Check out our paper for more details on our heuristic.
Open Dataset
To assess the quality of inputs suggested by our heuristic, and to facilitate future research, we create an open dataset of purpose statements with known ambiguities. For each function, we provide four variants of prompts.
- S (Signature)
- SP (S + Purpose statement)
- SP1 (SP + 1 functional example)
- SPx (SP + More functional examples)
Our dataset is publicly available here. We use this dataset to compare GuardRails with test cases generated by GitHub Copilot Chat. More details on this are available in our paper.
Owner
- Name: Mrigank Pawagi
- Login: mrigankpawagi
- Kind: user
- Location: Noida, Uttar Pradesh, India
- Website: https://mrigank.in/
- Twitter: mrigankpawagi
- Repositories: 0
- Profile: https://github.com/mrigankpawagi
I am a passionate learner who seeks to find innovative ways of leveraging science and technology to make people confident, secure, reliant, and aware.
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use or modify this tool, please cite it as below." authors: - family-names: "Pawagi" given-names: "Mrigank" orcid: "https://orcid.org/0009-0002-6169-4766" - family-names: "Kumar" given-names: "Viraj" orcid: "https://orcid.org/0000-0002-2252-0141" title: "GuardRails: Automated Suggestions for Clarifying Ambiguous Purpose Statements" version: 0.2.6 doi: 10.1145/3627217.3627234 date-released: 2023-12-10 url: "https://github.com/mrigankpawagi/GuardRails"
GitHub Events
Total
- Watch event: 4
- Delete event: 2
- Push event: 20
- Pull request event: 3
- Fork event: 1
- Create event: 2
Last Year
- Watch event: 4
- Delete event: 2
- Push event: 20
- Pull request event: 3
- Fork event: 1
- Create event: 2
Committers
Last synced: 10 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Mrigank Pawagi | m****i@g****m | 93 |
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 2
- Total pull requests: 4
- Average time to close issues: about 1 month
- Average time to close pull requests: less than a minute
- Total issue authors: 1
- Total pull request authors: 1
- Average comments per issue: 0.0
- Average comments per pull request: 0.0
- Merged pull requests: 2
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 3
- Average time to close issues: N/A
- Average time to close pull requests: less than a minute
- Issue authors: 0
- Pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 0.0
- Merged pull requests: 1
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- mrigankpawagi (2)
Pull Request Authors
- mrigankpawagi (6)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- 290 dependencies
- @types/glob ^8.0.1 development
- @types/mocha ^10.0.1 development
- @types/node 16.x development
- @types/vscode ^1.75.0 development
- @typescript-eslint/eslint-plugin ^5.49.0 development
- @typescript-eslint/parser ^5.49.0 development
- @vscode/test-electron ^2.2.2 development
- eslint ^8.33.0 development
- glob ^8.1.0 development
- mocha ^10.1.0 development
- typescript ^4.9.4 development
- node-fetch ^2.6.9
- node-html-parser ^6.1.5
- sqlite3 ^5.1.4
- Flask ==2.2.2
- Werkzeug ==2.2.2
- google.ai.generativelanguage *
- google.generativeai *
- multipledispatch *
- pyjail *
- python-dotenv *