data-driven-init
Data driven initialization for neural network models
Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (8.2%) to scientific vocabulary
Keywords
Repository
Data driven initialization for neural network models
Basic Info
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Topics
Metadata Files
README.md
Data driven initialization for neural network models
This repository contains the reference implementation of the IDEAL method to initialize the parameters of a neural network, using the training data to find adequate initial values for the model's weights and biases.
Repository structure
The initialization methods are implemented in ideal_init/initialization.py and can be used with PyTorch models. An example of how to use them with PyTorch Lightning modules can be found in ideal_init/lightning_model.py. These modules are used in all the examples in the remaining folders: tabular_classification, tabular_regression, image_classification and sequence_classification. These examples are Jupyter notebooks that can be directly run if PyTorch, scikit-learn and PyTorch Lightning are already installed.
Method performance
Although each notebook in the examples can be run independently from the others, we chose to visualize all the results in a single figure, using the minimalistic code in plot_training.ipynb. The results are presented in the following image, where IDEAL is shown in blue, the Kaiming He method in orange, the solid lines represent the mean of 10 similar experiments, and the shadows around the lines represent a 95% confidence interval.
Owner
- Name: Nicolas Rojas
- Login: nirogu
- Kind: user
- Company: Addi
- Website: https://nicolasrojas.me
- Repositories: 9
- Profile: https://github.com/nirogu
Applied mathematician and computer scientist.
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Rojas" given-names: "Nicolas" title: "Data driven initialization for neural network models" date-released: 2024-05-15 license: ISC url: "https://github.com/nirogu/data-driven-init"