Aggression Detection

Parties Involved

Institution: Munroe-Meyer Institute in the University of Nebraska Medical Center & Electrical and Computing Engineering Department in the University of Nebraska-Lincoln
Laboratory: Virtual Reality Laboratory & IMAGE-Signals Analysis Laboratory
Advisor: Dr. James Gehringer & Dr. Benjamin Riggan
Developer: Walker Arce

Motivation

This software repository contains all scripts developed to produce the results in our paper "Detecting Aggression in Clinical Treatment Videos". Due to patient confidentiality issues, the dataset used for this study cannot be shared.

Project Scripts

clip_export.py (180 LOC) - Generates a dataset using cometrics coded videos - Paths to folders and raw data must be added

cnn_transformer.py (290 LOC) - Contains Transformer model source - Contains recurrent model source - Contains unused model source

converttoreli.py (63 LOC) - Converts a primary session to a reliability session

dataloader_utils.py (572 LOC) - Contains multiple routines for processing the dataset and pickling it to speed up processing

datapal_inference.py (302 LOC) - Contains routines for inferencing using DataPal source files, is a precursor to future scripts

experiment_setup.py (655 LOC) - Contains configuration data for experiments including the main training and evaluation routines

full_dataset.py (529 LOC) - Generates dataset statistics

hog_testing.py (155 LOC) - Creates HOG detector

identify_patient.py (43 LOC) - Used to adapt an old dataset to new dataset format

keras_models.py (873 LOC) - This script was the main engine of the experiments, sets up filepaths and initiates training and evaluation

keypoint_detection.py (71 LOC) - Uses detectron2 to generate keypoint images

keypoint_utils.py (78 LOC) - Utilities for processing keypoint data

model_metrics.py (136 LOC) - Utilities for generating performance metrics on keras models

patientdatasetgenerator.py (401 LOC) - Tkinter user interface for generating annotations of patient and clinician using a stock YOLO model

patientdatasetgenerator_auto.py (436 LOC) - Tkinter user interface for automatically generating annotations of patient and clinician using a finetuned YOLO model

roc_creator.py (227 LOC) - Creates ROC curves used in this study

table_creator.py (72 LOC) - Creates tables used in this study

tag_lut.py (85 LOC) - Lookup table for tags used in this study

updatepaths.py (67 LOC) - Recreates YOLO dataset for training using the annotations generated by patientdataset_generator.py

vivit.py (129 LOC) - Instantiates ViViT model