Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.8%) to scientific vocabulary
Repository
Bayesian RG for NNFTs
Basic Info
- Host: GitHub
- Owner: xand-stapleton
- License: gpl-2.0
- Language: Jupyter Notebook
- Default Branch: main
- Homepage: https://arxiv.org/abs/2405.17538
- Size: 30.3 KB
Statistics
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 1
Metadata Files
README.md
Bayesian RG Flow in Neural Network Field Theories
Abstract
The Neural Network Field Theory correspondence (NNFT) is a mapping from neural network (NN) architectures into the space of statistical field theories (SFTs). The Bayesian renormalization group (BRG) is an information-theoretic coarse graining scheme that generalizes the principles of the Exact Renormalization Group (ERG) to arbitrarily parameterized probability distributions, including those of NNs. In BRG, coarse graining is performed in parameter space with respect to an information-theoretic distinguishability scale set by the Fisher information metric. In this paper, we unify NNFT and BRG to form a powerful new framework for exploring the space of NNs and SFTs, which we coin BRG-NNFT. With BRG-NNFT, NN training dynamics can be interpreted as inducing a flow in the space of SFTs from the information-theoretic ‘IR’ → ‘UV’. Conversely, applying an information-shell coarse graining to the trained network’s parameters induces a flow in the space of SFTs from the information-theoretic ‘UV’ → ‘IR’. When the information-theoretic cutoff scale coincides with a standard momentum scale, BRG is equivalent to ERG. We demonstrate the BRG-NNFT correspondence on two analytically tractable examples. First, we construct BRG flows for trained, infinite-width NNs, of arbitrary depth, with generic activation functions. As a special case, we then restrict to architectures with a single infinitely-wide layer, scalar outputs, and generalized cos-net activations. In this case, we show that BRG coarse-graining corresponds exactly to the momentum-shell ERG flow of a free scalar SFT in Euclidean space. Our analytic results are corroborated by a numerical experiment in which an ensemble of asymptotically wide NNs are trained and subsequently renormalized using an information-shell BRG scheme.
BibTeX Citation
If you use our code, please cite the paper
@misc{howard2024bayesian,
title={Bayesian RG Flow in Neural Network Field Theories},
author={Jessica N. Howard and Marc S. Klinger and Anindita Maiti and Alexander G. Stapleton},
year={2024},
eprint={2405.17538},
archivePrefix={arXiv},
primaryClass={hep-th}
}
Owner
- Name: Alexander Stapleton
- Login: xand-stapleton
- Kind: user
- Repositories: 1
- Profile: https://github.com/xand-stapleton
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Howard" given-names: "Jessica N." orcid: "https://orcid.org/0000-0001-8295-7727" - family-names: "Klinger" given-names: "Marc S." orcid: "https://orcid.org/0009-0001-7666-1185" - family-names: "Maiti" given-names: "Anindita" orcid: "https://orcid.org/0000-0002-4712-6626" - family-names: "Stapleton" given-names: "Alexander G." orcid: "https://orcid.org/0009-0009-6784-7779" title: "Bayesian RG Flow in Neural Network Field Theories" version: 1.0.0 # doi: TBD date-released: 2024-05-28 url: "https://arxiv.org/abs/2405.17538"
GitHub Events
Total
- Watch event: 1
Last Year
- Watch event: 1