https://github.com/coloquinte/overparam
Over-parameterization of linear and convolution layers in Pytorch
Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (3.4%) to scientific vocabulary
Repository
Over-parameterization of linear and convolution layers in Pytorch
Basic Info
- Host: GitHub
- Owner: Coloquinte
- License: mit
- Language: Python
- Default Branch: master
- Size: 9.77 KB
Statistics
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
overparam
Overparameterization replaces a linear layer with several larger layers during training, and collapses them at inference time. Although the two are mathematically equivalent, the networks with additional parameters tends to exhibit better training behaviour and final accuracy.
This code implements the ideas presented in ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks. It is based on the Pytorch code for weight normalization. The overparameterization is added as a forward hook, so that the performance penalty during training is minimal.
Usage
```python import torch.nn as nn from overparam import overparam, remove_overparam
m = nn.Conv2d(10, 20, kernelsize=3) overparam(m, expansion=4) ... removeoverparam(m) ```
Owner
- Name: Gabriel Gouvine
- Login: Coloquinte
- Kind: user
- Location: Edinburgh
- Company: AMD
- Repositories: 36
- Profile: https://github.com/Coloquinte
GitHub Events
Total
- Watch event: 1
Last Year
- Watch event: 1
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0