gradients
Build your deep learning algorithms with confidence
Science Score: 54.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: zenodo.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (15.6%) to scientific vocabulary
Keywords
Repository
Build your deep learning algorithms with confidence
Basic Info
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 4
- Releases: 1
Topics
Metadata Files
README.md
Build your deep learning models with confidence
Gradients provide a self consistency test function to perform gradient checking on your deep learning models. It uses centered finite difference approximation method to check the difference between analytical and numerical gradients and report if the check fails on any parameters of your model. Currently the library supports only PyTorch models built with custom layers, custom loss functions, activation functions and any neural network function subclassing AutoGrad.
Installation
pip install gradients
Package Overview
Optimizing deep learning models is a two step process:
Compute gradients with respect to parameters
Update the parameters given the gradients
In PyTorch, step 1 is done by the type-based automatic differentiation system torch.nn.autograd and 2 by the package implementing optimization algorithms torch.optim. Using them, we can develop fully customized deep learning models with torch.nn.Module and test them using Gradient as follows;
Activation function with backward
```python class MySigmoid(torch.autograd.Function):
@staticmethod
def forward(ctx, input):
output = 1/(1+torch.exp(-input))
ctx.save_for_backward(output)
return output
@staticmethod
def backward(ctx, grad_output):
input, = ctx.saved_tensors
return grad_output*input*(1-input)
```
Loss function with backward
```python class MSELoss(torch.autograd.Function):
@staticmethod
def forward(ctx, y_pred, y):
ctx.save_for_backward(y_pred, y)
return ((y_pred-y)**2).sum()/y_pred.shape[0]
@staticmethod
def backward(ctx, grad_output):
y_pred, y = ctx.saved_tensors
grad_input = 2 * (y_pred-y)/y_pred.shape[0]
return grad_input, None
```
Pytorch Model
python
class MyModel(torch.nn.Module):
def __init__(self,D_in, D_out):
super(MyModel,self).__init__()
self.w1 = torch.nn.Parameter(torch.randn(D_in, D_out), requires_grad=True)
self.sigmoid = MySigmoid.apply
def forward(self,x):
y_pred = self.sigmoid(x.mm(self.w1))
return y_pred
Check your implementation using Gradient
```python import torch from gradients import Gradient
N, Din, Dout = 10, 4, 3
Create random Tensors to hold inputs and outputs
x = torch.randn(N, Din) y = torch.randn(N, Dout)
Construct model by instantiating the class defined above
mymodel = MyModel(Din, Dout) criterion = MSELoss.apply
Test custom build model
Gradient(mymodel,x,y,criterion,eps=1e-8)
```
Owner
- Name: Saranraj Nambusubramaniyan
- Login: Saran-nns
- Kind: user
- Location: Chemnitz
- Repositories: 6
- Profile: https://github.com/Saran-nns
Comp. Neuroscience, Deep RL, State-space & Generative models. Professional account @sarannns
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Nambusubramaniyan" given-names: "Saranraj" orcid: "https://orcid.org/0000-0002-7314-0261" title: "gradients" version: 1.0.0 doi: 10.5281/zenodo.5176243 date-released: 2021-08-10 url: "https://github.com/Saran-nns/gradients"
GitHub Events
Total
Last Year
Issues and Pull Requests
Last synced: 7 months ago
All Time
- Total issues: 4
- Total pull requests: 1
- Average time to close issues: N/A
- Average time to close pull requests: about 2 hours
- Total issue authors: 1
- Total pull request authors: 1
- Average comments per issue: 0.0
- Average comments per pull request: 0.0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 1
- Average time to close issues: N/A
- Average time to close pull requests: about 2 hours
- Issue authors: 0
- Pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 0.0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- Saran-nns (4)
Pull Request Authors
- Saran-nns (2)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 1
-
Total downloads:
- pypi 48 last-month
- Total dependent packages: 0
- Total dependent repositories: 1
- Total versions: 3
- Total maintainers: 1
pypi.org: gradients
Gradient Checker for Custom built PyTorch Models
- Homepage: https://github.com/Saran-nns/gradients
- Documentation: https://gradients.readthedocs.io/
- License: OSI Approved :: MIT License
-
Latest release: 0.0.3
published over 4 years ago
Rankings
Maintainers (1)
Dependencies
- certifi ==2018.11.29
- cycler ==0.10.0
- kiwisolver ==1.0.1
- matplotlib ==3.0.3
- numpy ==1.16.2
- python-dateutil ==2.8.0
- pytz ==2018.9
- scipy ==1.2.1
- six ==1.12.0
- torch ==1.8.0
- wincertstore ==0.2
- numpy *
- actions/checkout v2 composite
- actions/setup-python v2 composite
- codecov/codecov-action v1 composite