tensorflow-engram

Engram Neural Networks (ENNs): Hebbian Memory-Augmented Recurrent Networks

Biologically-inspired memory for TensorFlow/Keras.

Add Hebbian/engram learning to your neural networks with just a few lines of code.

Overview

tensorflow-engram provides Keras layers, models, and utilities for building neural networks with biologically-inspired memory mechanisms, including Hebbian plasticity, engram-like trace formation, attention, and sparse memory recall. This enables powerful sequence modeling, few-shot learning, continual learning, and analysis of memory traces within modern deep learning pipelines.

• Seamless TensorFlow/Keras integration
• Engram layers: RNN cells and wrappers with memory banks, plastic synapses, and sparsity
• Hebbian learning: Fast local updates + gradient learning
• Attention and sparsity: Focuses on the most relevant memories
• Trace monitoring: Visualize engram and memory trace evolution
• Ready-to-use models for classification and regression

Tensorflow-Engram is currently in development and may not yet be ready for production use. We are actively seeking contributors to help us improve the package and expand its capabilities. If you are interested in contributing, please see our contributing guide.

TODO:

• Add unit tests
• Generate doc pages with Sphinx
• Possibly rename repo?

Installation

bash pip install tensorflow-engram

Or install using conda:

bash conda install -c danielathome19 tensorflow-engram

Requirements:

• Python 3.12+
• TensorFlow 2.19+
• Keras 3.10+
• numpy, seaborn, matplotlib, pandas (for utilities and plotting)

Quickstart

Example: MNIST Classification with Engram Memory

```python import numpy as np from tensorflow.keras.datasets import mnist from tensorflow.keras.utils import tocategorical from sklearn.modelselection import traintestsplit from tensorflowengram.models import EngramClassifier from tensorflowengram.utils import HebbianTraceMonitor, plothebbiantrace

Prepare data

(xtrain, ytrain), (xtest, ytest) = mnist.loaddata() xtrain = xtrain.astype('float32') / 255.0 xtest = xtest.astype('float32') / 255.0 ytrain = tocategorical(ytrain, 10) ytest = tocategorical(ytest, 10) xtrain = xtrain.reshape(-1, 28, 28) xtest = xtest.reshape(-1, 28, 28) xtrain, xval, ytrain, yval = traintestsplit(xtrain, ytrain, testsize=0.1)

Build model

model = EngramClassifier( inputshape=(28, 28), numclasses=10, hiddendim=128, memorysize=64, returnstates=True, hebbianlr=0.05, )

Monitor Hebbian trace during training

tracecallback = HebbianTraceMonitor(xtrain[:32], log_dir=None)

model.compile( optimizer='adam', loss='categoricalcrossentropy', metrics=['accuracy'] ) model.fit( xtrain, ytrain, batchsize=128, epochs=10, validationdata=(xval, yval), callbacks=[tracecallback] )

Visualize trace evolution

plothebbiantrace(trace_callback) ```

Features

• EngramCell: Biologically-inspired RNN cell with memory banks and Hebbian plasticity.
• EngramNetwork: High-level Keras Model for sequence modeling.
• Attention Layer: Optional attention mechanism for sequence summarization.
• Trace Monitoring: Inspect and visualize memory trace evolution with built-in callbacks and plotting utilities.

API Highlights

Layers

• EngramCell: Biologically-inspired RNN cell with memory banks, Hebbian trace, and sparsity regularization.

• Engram: Wrapper for Keras models/networks using EngramCell.

• EngramAttentionLayer: Optional attention over sequence outputs.

Models

• EngramNetwork: General-purpose sequence model with configurable memory and plasticity.

• EngramClassifier: Factory function for classification tasks.

• EngramRegressor: Factory for regression tasks.

Utilities

• HebbianTraceMonitor: Keras callback for logging and visualizing Hebbian traces.

• plot_hebbian_trace: Quick plotting of trace evolution and statistics.

How It Works

• Memory Bank: Persistent, learnable memory vectors (engrams), updated via gradient descent.

• Hebbian Trace: Rapidly updated, plastic component reflecting short-term memory, updated via local Hebbian learning.

• Attention/Recall & Sparsity: Memories are retrieved by attention (cosine similarity + softmax), but with sparsity constraints so only a few are activated per input—mimicking efficient biological memory recall.

• Trace Visualization: Built-in tools to monitor and understand the dynamics of memory during training.

Advanced Usage

You can customize the cell and models for your own tasks:

```python from tensorflow_engram.layers import EngramCell from tensorflow.keras.layers import RNN, Input from tensorflow.keras.models import Model

cell = EngramCell(hiddendim=64, memorysize=32) inputs = Input(shape=(None, 16)) rnnlayer = RNN(cell, returnsequences=True) outputs = rnn_layer(inputs) model = Model(inputs, outputs) ```

License

Tensorflow-Engram is licensed under the BSD-3 License. See the LICENSE file for more information.

Citation

If you use this code for your research, please cite this project as:

bibtex @software{Szelogowski_tensorflow_engram_2025, author = {Szelogowski, Daniel}, doi = {10.48550/arXiv.2507.21474}, license = {BSD-3-Clause}, month = {jul}, title = {{tensorflow-engram: A Python package for Engram Neural Networks, adding biologically-inspired Hebbian memory and engram layers to TensorFlow/Keras models, supporting memory traces, plasticity, attention, and sparsity for neural sequence learning.}}, url = {https://github.com/danielathome19/Engram-Neural-Network}, version = {0.1.0}, year = {2025} }

or as the corresponding research paper:

bibtex @misc{Szelogowski_Simulation_of_Neural_Responses_Using_OI_2024, author = {Szelogowski, Daniel}, doi = {10.48550/arXiv.2507.21474}, month = {jul}, title = {{Hebbian Memory-Augmented Recurrent Networks: Engram Neurons in Deep Learning}}, url = {https://github.com/danielathome19/Engram-Neural-Network}, year = {2025} }