Human Activity Recognition

This notebook uses data from, and is an adaptation of, . It demonstrates the process of creating a basic motion sensing activity classifier using Keras and TensorFlow Lite, tailored for deployment in the .

Import TensorFlow

We use TensorFlow 2.19.0.

python import tensorflow as tf print(tf.version.VERSION)

2.19.0

Loading the dataset

The dataset is composed of 3-axis acceleration values stored in .csv text format. Each individual file corresponds to a capture characterised by it's containing folder name.

Download and extract the dataset:

python !wget -nc https://github.com/STMicroelectronics/stm32ai/raw/master/AI_resources/HAR/dataset.zip

python !unzip -n dataset.zip

Load the dataset into memory:

```python import glob import numpy as np

Load data into memory

labels = ['stationary', 'walking', 'running'] xrecordings = [] yrecordings = [] recordingsfilenames = [] for i, label in enumerate(labels): filenames = glob.glob('dataset/' + label + '/*.csv') for filename in filenames: data = np.loadtxt(filename, delimiter=',') xrecordings.append(data) yrecordings.append(i) recordingsfilenames.append(filename)

xrecordings = np.array(xrecordings).reshape(len(xrecordings), -1, 3) yrecordings = np.array(y_recordings)

print(xrecordings.shape) print(yrecordings.shape) ```

Exploring the dataset

To verify that the dataset has been loaded correctly, let's display and plot a few captures at random.

```python import random import matplotlib.pyplot as plt %matplotlib inline

Plot some captures

random.seed(10) uniquerands = random.sample(range(len(xrecordings)), 10) plt.figure(figsize=(18, 10)) for i, n in enumerate(uniquerands): plt.subplot(5, 2, i + 1) plt.margins(x=0, y=-0.25) plt.plot(xrecordings[n]) plt.ylim(-4000, 4000) # 4000 mg acc. range plt.title(recordingsfilenames[n].split('/')[-1]) plt.tightlayout() plt.show() ```

Frame data

Each capture is framed into smaller windows. The window size will be the input dimension for the neural network. Optional overlapping can be added to further increase the number of points in our data for training and testing.

Here we are dividing the dataset into 10 second frames (260 samples at 26 Hz):

```python import numpy as np

def frame(x, framelen, hoplen): '''Slice a 3D data array into (overlapping) frames.

Example
--------
>>> x = np.array([[0, 1, 2],
                  [10, 11, 12],
                  [20, 21, 22],
                  [30, 31, 32],
                  [40, 41, 42],
                  [50, 51, 52],
                  [60, 61, 62]])
>>> frames = x.frame(x, 3, 2)
>>> x.shape
(7, 3)
>>> frames.shape
(3, 3, 3)
'''

assert(x.shape == (len(x), 3))
assert(x.shape[0] >= frame_len)
assert(hop_len >= 1)

n_frames = 1 + (x.shape[0] - frame_len) // hop_len
shape = (n_frames, frame_len, x.shape[1])
strides = ((hop_len * x.strides[0],) + x.strides)
return np.lib.stride_tricks.as_strided(x, shape=shape, strides=strides)

xframes = [] yframes = [] for i in range(xrecordings.shape[0]): # frames = frame(xrecordings[i], 26, 26) # no overlap frames = frame(xrecordings[i], 260, 26) # 10 second frame, 1 second overlap -> 10 % overlap xframes.append(frames) yframes.append(np.full(frames.shape[0], yrecordings[i]))

print(np.array(xframes).shape) xframes = np.concatenate(xframes) yframes = np.concatenate(yframes) print(xframes.shape)

Each output label is an integer between 0 and 2:

print(y_frames.shape) print(labels) ```

Preprocessing the dataset

To improve the model's performance, it is recommended to perform some kind of pre-processing on the input data before feeding them to the neural network model. In this example, data is normalized into floating point values between -1.0 and 1.0 by dividing them by the sensor's full range (4000 mg).

```python

Normalize input data between [-1;1]

xframesnormed = x_frames / 4000 ```

Preparing the dataset (train, test, split)

The dataset is split into a train and test set. The data is first shuffled and then split with 25% reserved for model evaluation and testing purposes.

```python from sklearn.modelselection import traintest_split

xtrain, xtest, ytrain, ytest = traintestsplit( xframesnormed, yframes, testsize=0.25)

print("Trainning samples:", xtrain.shape) print("Testing samples:", xtest.shape) print(xtrain.shape) # Should be (numsamples, 260, 3) print(x_train[0].shape) # Should be (260, 3) ```

Trainning samples: (2001, 260, 3)
Testing samples: (667, 260, 3)
(2001, 260, 3)
(260, 3)

Creating the model

Let's create a classifier model using Keras.

Build the tf.keras.Sequential model by stacking layers. Choose an optimizer and loss function for training:

```python

Conv1D based model

model = tf.keras.models.Sequential([ tf.keras.layers.Conv1D(filters=16, kernelsize=3, activation='relu', inputshape=(260, 3)), #input adjusted tf.keras.layers.Conv1D(filters=8, kernelsize=3, activation='relu'), tf.keras.layers.Dropout(0.5), tf.keras.layers.Flatten(), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dense(3, activation='softmax') ]) model.compile(optimizer='adam', loss='sparsecategoricalcrossentropy', metrics=['accuracy']) model.fit(xtrain, ytrain, epochs=30) testloss, testacc = model.evaluate(xtest, y_test, verbose=2)

print("Test loss:", testloss) print("Test acc:", testacc) model.summary() ```

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv1d_10 (Conv1D)              │ (None, 258, 16)        │           160 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv1d_11 (Conv1D)              │ (None, 256, 8)         │           392 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_5 (Dropout)             │ (None, 256, 8)         │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_5 (Flatten)             │ (None, 2048)           │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_10 (Dense)                │ (None, 64)             │       131,136 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_11 (Dense)                │ (None, 3)              │           195 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 395,651 (1.51 MB)

 Trainable params: 131,883 (515.17 KB)

 Non-trainable params: 0 (0.00 B)

 Optimizer params: 263,768 (1.01 MB)

Creating the confusion matrix

```python import matplotlib.pyplot as plt import seaborn as sns %matplotlib inline

Ypred = model.predict(xtest) ypred = np.argmax(Ypred, axis=1) confusionmatrix = tf.math.confusionmatrix(ytest, ypred)

plt.figure() sns.heatmap(confusionmatrix, annot=True, xticklabels=labels, yticklabels=labels, cmap=plt.cm.Blues, fmt='d', cbar=False) plt.tightlayout() plt.ylabel('True label') plt.xlabel('Predicted label') plt.show() print("Unique ytest labels:", np.unique(ytest)) print("Labels list:", labels) ```

Creating the TFLite model

```python converter = tf.lite.TFLiteConverter.fromkerasmodel(model) tflite_model = converter.convert()

with open('model.tflite', 'wb') as f: f.write(tflite_model) ```

Third-Party Notices

This project includes data and adapted content from:

STMicroelectronics – Human Activity Recognition notebook
Licensed under the BSD 3-Clause License
https://opensource.org/licenses/BSD-3-Clause

Original source:
https://github.com/STMicroelectronics/stm32ai-wiki/blob/master/AIresources/HAR/HumanActivity_Recognition.ipynb