See the Workshop Webpage for the context of this tutorial.
See the Workshop Hands-on Instructions for initial installation steps.

Introduction

The gap between what is achieve in the computer vision field, and the computer vision tools used in other fields, exists because of the lack of communication between the two fields. Thanks to this workshop and the proposed hand-on session, we hope to fill this gap in an funny and interactive way.

In this tutorial, we shall solve a simple classification task with one provided dataset, and if you have time, your own dataset! Let's hope you can do the same with a more serious scientific question in your future projects!

Installation

Before coming to the workshop, make sure to set your computer. 64bit computer is requiered. GPU and CUDA is advised but not mandatory. For simplicity, we use anaconda to install the requiered librairies.

In your termianl (or conda prompt) opened at the right location (folder dedicated to the workshop - use cd to navigate in your folders), follow the next steps:

``` bash

Clone baseline repo via command line using git or manually using the browser: download zip file, extract it in appropriate location and open a terminal in tuto_classification folder

Install git if not already installed

conda install -c anaconda git

git function for cloning

git clone https://github.com/ccp-eva/tuto_classification.git

cd function to go to your directory

cd tuto_classification ```

You should have downloaded the github repo. This repo should contain several python file. In order to all have the same installation environment, we will use conda.

Replicate environment

With .yml files (preferred)

This environment is the same than the one provided in the Gaze Workshop. If already installed, you can just activate if by running: conda activate env_workshop. If not, running the next lines in this section will install the required environment.

For all platforms, the provided .yml file should help you replicated the environment. Conda shall do the rest to make it work.

``` bash

Create the conda environment

conda env create -f environment.yml

Activate the conda environment

conda activate env_workshop ```

From scratch (if previous did not work)

For linux based platform

``` bash

Create a new environment with python 3.7 for compatibility with the different librairies

conda create -n envtuto python=3.7 conda activate envtuto

Install pytorch

conda install pytorch torchvision torchaudio cpuonly -c pytorch

Install opencv and matplotlib

conda install -c conda-forge opencv matplotlib

Install scikit learn

conda install -c anaconda scikit-learn ```

For MAC

``` bash

Create a new environment with python 3.9 for compatibility with the different librairies

conda create -n envtuto python=3.9 conda activate envtuto

Install pytorch

conda install pytorch torchvision torchaudio -c pytorch

Install opencv and matplotlib

conda install -c conda-forge opencv matplotlib

Install scikit learn

conda install -c anaconda scikit-learn ```

For Windows

``` bash

Create a new environment with python 3.9 for compatibility with the different librairies

conda create -n envtuto python=3.9 conda activate envtuto

Install pytorch

conda install pytorch torchvision torchaudio cpuonly -c pytorch

Install opencv and matplotlib

conda install -c conda-forge opencv matplotlib

Install scikit learn

conda install -c anaconda scikit-learn ```

Run the code

The next steps may be done before the workshop and any bug/difficulties may be reported to the organizers.

``` bash

Extract frames from videos

python 1createdatabase.py

Split your databse

python 2splitdatabase.py

Run classification on you database split framewisely

python 3cnnclassification.py mydatasetframewise_split

Run classification on you database split videowisely

python 3cnnclassification.py mydatasetvideowise_split ```

These steps should create several folders: - a log folder for script 1 and 2. - a few images to represent your datasets. - cnnclassificationoutput folder for script 3. It should contain to some depth models for each call of the function. The latest should contain the weights of the model your just trained on the given dataset, and some images representing its performance.

Finally, use one of the output folder of script 3 to run the demo app. It shall use your camera and indicate the infered class of your model. Replace [path_of_you_trained_model_folder] by the path of the folder containing the weights of your model (should be something like cnn_classification_output/my_dataset_framewise_split/weighted_loss_False/2022-06-13_18-47-30/).

``` bash

Run your app

python 4runapp.py [pathofyoutrainedmodel_folder] --show-demo ```

You can also run the app with an external video:

``` bash

Run your app

python 4runapp.py [pathofyoutrainedmodelfolder] --video-input videodemo.mp4 --show-demo ```

Commun issues

• images are 90 clockwise rotated: not an error. Internal rotation of the device is not taken into with some librairies.
• not enough space when running script 3 - happen when cpu is not powerful enough. Solved by lowering the batch_size variable l.247 to 10 or even 2.
• problem with allocation of process in script 3 - happen when resctricted in cpu power/number. Solved by lowering the num_workers variable l.248 to 0.
• problem with logging and special characters. You may ignore this error is the code continues. The log file will just be incomplete.
• Mac users with M1 chip may not be able to install properly opencv using the classical anaconda version. Instead, you can try to re-install anaconda with a version develloped by the community: bash # Remove previous environment conda env remove -n env_workshop # Unistall previous anaconda version conda install anaconda-clean anaconda-clean -yes rm -rf ~/anaconda3 rm -rf ~/.anaconda_backup # Reinstall anaconda with Hoemebrew ## Install homebrew /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" ## Install anaconda community version brew install miniforge conda init zsh # Redo the first steps: ## Create the conda environment conda env create -f environment.yml ## Activate the conda environment conda activate env_workshop Then you may try again to run the code.
• Import error with PIL: bash ImportError: cannot import name 'PILLOW_VERSION' from 'PIL' [PATH] Solved by running: bash conda install -c anaconda pillow=6.1 # Or if does not work use pip install Pillow==6.1
• Error showing image opencv when running script 4 with camera. bash cv2.error: OpenCV(3.4.2) /tmp/build/80754af9/opencv-suite_1535558553474/work/modules/highgui/src/window.cpp:632: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage' The following solution may lead with no longer be able to process videos. Meaning run script 1 and script 4 with a video. Solution: by running (): bash conda remove opencv # If python version > 3.6 conda install python=3.6 conda install -c menpo opencv3 The torch version may have been affected. Therefore you will have to re-run script 3 before be able to use script 4.
• CUDA Error. There may come from various reason. Then use cpu in script 3 and 4 by setting the device variable to cpu. python 3_cnn_classification.py my_dataset_framewise_split --device cpu python 4_run_app.py [path_of_you_trained_model_folder] --device cpu
• If you have an error looking like this OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized. in script 3 and 4, please switch the 2 first lines in the 2 scripts like this:

import torch from utils import *

Further steps

Further steps will be conducted at the workshop. If time allows it, we may create our own dataset and our own task we would like to solve.

Thank you for your participation!