osculari
Exploring and interpreting pretrained deep neural networks.
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 1 DOI reference(s) in README -
✓Academic publication links
Links to: zenodo.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.8%) to scientific vocabulary
Keywords
Repository
Exploring and interpreting pretrained deep neural networks.
Basic Info
- Host: GitHub
- Owner: ArashAkbarinia
- License: mit
- Language: Python
- Default Branch: main
- Homepage: https://osculari.readthedocs.io/
- Size: 2.23 MB
Statistics
- Stars: 6
- Watchers: 3
- Forks: 0
- Open Issues: 0
- Releases: 3
Topics
Metadata Files
README.md
osculari
Exploring artificial neural networks with psychophysical experiments.
Overview
The osculari package provides an easy interface for different techniques to explore and interpret
the internal presentation of deep neural networks.
- Supporting the following pretrained models:
- All classification and segmentation networks from PyTorch's official website.
- All OpenAI CLIP language-vision models.
- All Taskonomy networks.
- Managing convolution and transformer architectures.
- Allowing to readout the network at any given depth.
- Training a linear classifier on top of the extract features from any network/layer.
- Experimenting with 2AFC and 4AFC paradigms.
At a granular level, Kornia is a library that consists of the following components:
| Module | Description | |-----------------------------------------------------------------------------------------|----------------------------------------------------------------------------------| | osculari | Open source library to explore and interpret pretrained deep neural networks. | | osculari.datasets | A module to create datasets and dataloaders to train and test linear probes. | | osculari.models | A module to readout pretrained networks and add linear layers on top of them. | | osculari.paradigms | A module to implement psychophysical paradigms to experiment with deep networks. |
Installation
From pip
bash
pip install osculari
Alternative installation options
### From source with symbolic links: ```bash pip install -e . ``` ### From source using pip: ```bash pip install git+https://github.com/ArashAkbarinia/osculari ```Examples
Please check the example page of our documentation with many notebooks that can also be executed on Google Colab.
Quick start
### Pretrained features Let's create a linear classifier on top of the extracted features from a pretrained network to perform a binary classification task (i.e., 2AFC – two-alternative-force-choice). This is easily achieved by calling the `paradigm_2afc_merge_concatenate` from the `osculari.models` module. ``` python architecture = 'resnet50' # network's architecture weights = 'resnet50' # the pretrained weights img_size = 224 # network's input size layer = 'block0' # the readout layer readout_kwargs = { 'architecture': architecture, 'weights': weights, 'layers': layer, 'img_size': img_size, } net_2afc = osculari.models.paradigm_2afc_merge_concatenate(**readout_kwargs) ``` ### Datasets The `osculari.datasets` module provides datasets that are generated randomly on the fly with flexible properties that can be dynamically changed based on the experiment of interest. For instance, by passing a `appearance_fun` to the `ShapeAppearanceDataset` class, we can dynamically merge foreground masks with background images to generate stimuli of interest. ```python def appearance_fun(foregrounds, backgrounds): # implementing the required appearance (colour, texture, etc.) on foreground and merging # to background. return merged_imgs, ground_truth num_samples = 1000 # the number of random samples generated in the dataset num_imgs = net_2afc.input_nodes # the number of images in each sample background = 128 # the background type dataset = osculari.datasets.geometrical_shapes.ShapeAppearanceDataset( num_samples, num_imgs, img_size, background, appearance_fun, unique_bg=True, transform=net_2afc.preprocess_transform() ) ``` ### Linear probe The `osculari.paradigms` module implements a set of psychophysical paradigms. The `train_linear_probe` function trains the network on a dataset following the paradigm passed to the function. ```python # experiment-dependent function to train on an epoch of data epoch_fun = osculari.paradigms.forced_choice.epoch_loop # calling the generic train_linear_probe function training_log = osculari.paradigms.paradigm_utils.train_linear_probe( net_2afc, dataset, epoch_fun, './osculari_test/' ) ``` ### Psychophysical experiment The `osculari.paradigms` module also implements a set of psychophysical experiments similar to the experiments conducted with human participants. In this example, we use the `staircase` function to gradually measure the network's sensitivity. ```python # experiment-dependent function to test an epoch of data test_epoch_fun = osculari.paradigms.forced_choice.test_dataset # the test dataset implementing desired stimuli. class TestDataset(TorchDataset): def __getitem__(self, idx): return stimuli test_log = osculari.paradigms.staircase( net_2afc, test_epoch_fun, TestDataset(), low_val=0, high_val=1 ) ```Contribution
We welcome all contributions to the project that extend or improve code and/or documentation! Please read the CONTRIBUTING page and follow the Code of Conduct.
Owner
- Name: Arash Akbarinia
- Login: ArashAkbarinia
- Kind: user
- Website: https://arashakbarinia.github.io/
- Repositories: 14
- Profile: https://github.com/ArashAkbarinia
Citation (CITATION.cff)
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!
cff-version: 1.2.0
title: >-
Osculari: a Python package to explore artificial neural networks with psychophysical experiments
message: >-
If you use this software, please cite it using the metadata from this file.
type: software
authors:
- given-names: Arash
family-names: Akbarinia
orcid: 'https://orcid.org/0000-0002-4249-231X'
identifiers:
- type: doi
value: 10.5281/zenodo.10214006
repository-code: 'https://github.com/ArashAkbarinia/osculari'
license: MIT
version: v0.0.4
date-released: '2023-12-21'
GitHub Events
Total
- Watch event: 1
Last Year
- Watch event: 1
Packages
- Total packages: 1
-
Total downloads:
- pypi 21 last-month
- Total dependent packages: 0
- Total dependent repositories: 0
- Total versions: 4
- Total maintainers: 1
pypi.org: osculari
Open source library to explore artificial neural networks with psychophysical experiments.
- Homepage: https://github.com/ArashAkbarinia/osculari
- Documentation: https://osculari.readthedocs.io/en/latest
- License: MIT License
-
Latest release: 0.0.4
published about 2 years ago
Rankings
Maintainers (1)
Dependencies
- clip-openai *
- numpy *
- torch *
- torchvision *
- sphinx *
- sphinx-autobuild *
- sphinx-typlog-theme *
- sphinx_rtd_theme *