Releases | Open Source Science

Sports2D - Fixed GCV filtering + Expired OpenMMLab certificate

Fixed gcv filtering (careful if series too short: noise can be considered as signal -> no filtering. See this conversation: https://stackoverflow.com/a/79740481/12196632, and this issue: https://github.com/scipy/scipy/issues/23472
Temporarily ignore SSL certificate verification to handle OpenMMLab's expired certificate

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.17...v0.8.18

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 8 months ago

Sports2D - More filtering options (based on Pose2Sim)

Sports2D now uses the Pose2Sim filtering code. This is better, as when new features might be added, they will only need to be brought in one place (in the same way as the Sports2D scaling and IK code is dependent on Pose2Sim)

Added optional Hampel filter for outlier rejection, to be run befur further filtering methods. Rejects outliers that are outside of a 95% confidence interal from the median in a sliding window of size 7.
Added GCV spline filter. Automatically determines optimal parameters for each point, which is good when some move faster than others (e.g., fingers vs hips). User can make it biased towards more smoothing (>1) or more fidelity to data (<1) by adjusting the smoothingfactor. Might severely under or over-smooth due to numerical precision issues, so use with caution. I could not figure out a way to make it more robust, even after normalizing time and/or the observed y values. Feel free to intervene! Acts as a Butterworth filter if cutoff_frequency is set to an integer instead of 'auto'.
Fixed Kalman filter. Simplified version, the user says how much more they trust triangulation results (measurements), than the assumption of constant acceleration (process)
The available filters are now: Butterworth, Kalman, GCV-spline, LOESS, Median, Gaussian, Butterworth on speed

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.16...v0.8.17

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 8 months ago

Sports2D - trim video around valid frames

trimmed video around valid frames

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 8 months ago

Sports2D - visible side (auto or manual) was not taken into account when flip_left_side = False

visibleside (auto or manual) was not taken into account when flipleft_side = False (which is required for sprinting).

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 9 months ago

Sports2D - Various small fixes

Post-processed angles better unwrapped and kept between -180 and 180°
Displayed image is now in the correct ratio and as large as the screen allows
added "sectionstokeep" argument (all, largest, first, last)
Image indices used to be shifted back 1 frame
when clicking the persons to analyze, the first annotation never disappeared
no tracking when tracking mode is none (instead of sports2d tracking)
Added clarification on angles in readme (flipleftright, correctsegmentangleswithfloor_angle)

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.13...v0.8.14

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 9 months ago

Sports2D - stop freezing person at beginning and end of clips

Here is how things work now for the curation of pose estimation (italics are new elements): - keypoint pose is replaced with a gap if the confidence is lower than 0.3; the full person is ignored if the average keypoint confidence is below 0.5; and the person is also ignored if there are fewer than 3 keypoints with confidence above 0.3 - gaps are interpolated, unless they are larger than 10 frames - if the gaps are too large, the person freezes until we've got good values again - the person is not frozen anymore in the beginning and end of the clip: the output pose starts with the first chunk of 10 valid frames, and ends with the last one - the remaining data is filtered with a butterworth 10 Hz filter

Also added plot name for angles (with mac support)

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.10...v0.8.13

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 9 months ago

Sports2D - v0.8.12

Here is how things work now for the curation of pose estimation (italics are new elements): - keypoint pose is replaced with a gap if the confidence is lower than 0.3; the full person is ignored if the average keypoint confidence is below 0.5; and the person is also ignored if there are fewer than 3 keypoints with confidence above 0.3 - gaps are interpolated, unless they are larger than 10 frames - if the gaps are too large, the person freezes until we've got good values again - the person is not frozen anymore in the beginning and end of the clip: the output pose starts with the first chunk of 10 valid frames, and ends with the last one - the remaining data is filtered with a butterworth 10 Hz filter

Also added plot name for angles (withmac support

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.10...v0.8.12

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 9 months ago

Sports2D - Support plotting on macOS

What's Changed

prevent error for plot when using mac by @hunminkim98 in https://github.com/davidpagnon/Sports2D/pull/29

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.10...v0.8.11

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 9 months ago

Sports2D - use_simple_model argument (faster but no muscles or flexible spine)

See this Pose2Sim release: https://github.com/perfanalytics/pose2sim/releases/tag/v0.10.25 Make sure you also upgrade Pose2Sim with pip install pose2sim --upgrade

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.9...v0.8.10

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 10 months ago

Sports2D - Little fixes for multi-threading and others

Multi-threading: Waits a bit and retry after 3 ms if the setup of pose_tracker fails
Separated the setup of pose_model, ModelClass, mode into a new function

- Supported CUSTOM pose model

What's Changed

Fix incorrect command for usage of WholeBody model by @arghhhhh in https://github.com/davidpagnon/Sports2D/pull/26

New Contributors

@arghhhhh made their first contribution in https://github.com/davidpagnon/Sports2D/pull/26

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.7...v0.8.8

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 10 months ago

Sports2D - added feet_on_floor parameter and fixed whole_body pose estimation

On marker augmentation: - added feetonfloor parameter - fixed whole_body pose estimation

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.6...v0.8.7

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 11 months ago

Sports2D - Fixed 'on_click' ordering method

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.5...v0.8.6

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 12 months ago

Sports2D - Small fix for new Pose2Sim version

Just a few tiny edits for compatibility with new pose2sim marker augmentation, needs to be upgraded though
Updated continuous integration and automatic pypi publication upon new release based on Pose2Sim

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.4...v0.8.5

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 12 months ago

Sports2D - v0.8.4

fixed error when calculate_angles to false
new ordering method: largest and smallest size
on_click as default ordering method
fixed pyproject file to comply with latest PEP 639

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.3...v0.8.4

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon 12 months ago

Sports2D - does not store all frames in memory if they are not saved or used for ordering

Does not store all frames in memory if they are not saved or used for ordering Avoids out-of-memory errors on long captures.

Need to not save images or video, and to order persons with a method other than "onclick", for example with: `--saveimg false --savevid false --personorderingmethod highestlikelihood`

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.2...v0.8.3

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - deepcopy of pose_model to make sure keypoint_ids are not overwritten

Very specific issue, when it is run for several trials from within python, e.g.:

```python import toml from pathlib import Path from Sports2D import Sports2D

video_names = ['000000.mov', '000001.mov']

configpath = Path(Sports2D.file).parent / 'Demo/Configdemo.toml' configdict = toml.load(configpath) for vid in videonames: configdict.get("base").update({"videoinput":vid}) Sports2D.process(configdict) ```

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.1...v0.8.2

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - Fixed marker augmentation

Fixed marker augmentation:
- made sure the right trc data is taken
- failed when nans in trc: using ffill and bfill
minor other fixes (see release notes)

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.8.0...v0.8.1

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - Click and select the persons to analyze

/!\ Warning! Make sure that the last Pose2Sim version is installed with pip install pose2sim -U

1. Set the number of persons to analyze and their ordering method: See https://github.com/davidpagnon/Sports2D?tab=readme-ov-file#select-the-persons-you-are-interested-in Specify the number of persons you want to analyze and post-process (set 'all' if you want them all) Order them with one of these 6 methods: - 'onclick', - 'highestlikelihood', - 'greatestdisplacement', - 'leastdisplacement', - 'firstdetected', - 'lastdetected' Note that the person with reference height will be the first one coming out of this ordering. Among other advantages, this will make it easier to set the height and masses of the each person in the right order. I tried to make the "on_click" UI look consistent with the synchronization one by HunMin.

Beware of the changes in the parameters: - New: nbpersonstodetect = 'all' # int or 'all' # Limiting or not the number of persons to be analyzed - New: personselectionmethod = 'onclick' # 'onclick', 'highestlikelihood', 'greatestdisplacement', 'leastdisplacement', 'firstdetected', or 'lastdetected' - New: firstpersonheight = 1.76 - Removed: multiperson is removed (replaced by nbpersonstodetect) - Removed: pxtompersonheight (becomes firstpersonheight) - Removed: pxtomfrompersonid (first selected person will be chosen) - Changed: the 'project' and 'process' sections of the configuration file have been replaced by 'base'

Note that the default visible_side order has changed and is now 'auto', 'front', 'none'

Person_selection

2. Others: - Only the selected persons are saved (images, videos, trc, mot), with post-processed angles. - Fixed time index and image saves with loadtrcpx - Fixed angle computation with wholebody posemodel - Handled warning in computefloorline when a line is full of nan - Deleted setup.cfg and setup.py to switch to pyproject.toml

Also, the code is becoming kind of a mess... What I should do is rewrite it all, but that requires time I do not have.

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - Other minor edits

updated instructions and presentation video
remove all the functions that were already defined in Pose2Sim
doik and useaugmentation to false by default
no opensim import if do_ik is false
no markeraugmentation import if use_augmentation is false
package version assigned dynamically from setup.cfg

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.7.2...v0.7.3

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - Minor improvements

Minor fixes: - Fixed timerange - Fixed loadtrc_px - Fixed tests

Updated Pose2Sim model DOFs of the arms to avoid the elbow locking up when the shoulder is overly rotated

What's Changed

fixed tests by @davidpagnon in https://github.com/davidpagnon/Sports2D/pull/23

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.7.0...v0.7.2

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - Supported marker augmentation and inverse kinematics + others

Depth values in trc files adjusted based on neutral position for the sagittal plane (left and right side) and frontal plane (front and back side)
Supported marker augmentation and inverse kinematics via Pose2Sim
c3d export
renamed a bunch of parameters
fixed wholebodywrist model
Fixed "load_trc" option
Updated doc

What's Changed

Supported marker augmentation and inverse kinematics + others by @davidpagnon in https://github.com/davidpagnon/Sports2D/pull/22

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.6.3...v0.7.0

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - c3d export + actual 3D trc files

trc Z values not set to zeros but to coherent values based on visible side
c3d export
renamed a bunch of parameters

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - DeepSort tracking and better skeleton visualization

Skeleton visualization: A bounding-box color per person. Left and right limbs in a different color to better spot swaps. Keypoint color based on confidence.
Support DeepSort tracking across frames

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.6.1...v0.6.2

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - Any detection and pose model can be used + more

install custom rtmlib version (see https://github.com/Tau-J/rtmlib/pull/45) with:
- a new more general Custom class
- PoseTracker working with single-stage models (without detection)
- working NMS for RTMO
- fixed wholebody solution in balanced mode (see https://github.com/perfanalytics/pose2sim/issues/143#issuecomment-2479075518)
More versatile "mode" argument in sports2d: Any person detection and/or pose estimation model can now be used
Body, bodywithfeet, whole_body now supported
Fixed Hip, Shoulder, Trunk angles when Hip and Neck keypoints not available
-13 cm offset in floor line when feet are not available
Wrist angles supported in WholeBody model
Segment angles corrected with regards to the floor angle
Added hand, face, animal, wholebody skeletons (to match Pose2Sim skeletons.py)
Many fallback mechanisms

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 1 year ago

Sports2D - v0.5.6

solved various edge cases when multiperson false and load_trc
unwrapping angles does not work when nans in array: masked them
made trc and vid with real framerate (fps*slowmo) is finally cleaner
handled nanmean warning in av_speeds

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.5.5...v0.5.6

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Important improvements

better floor angle and axis calculation
unwrap angles to prevent jumps
faster when load_trc

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.5.4...v0.5.5

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - v0.5.4

custom_logging in case this is meant to be integrated in an API that already has logging

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.5.3...v0.5.4

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Other edge cases fixed

Took slowmo into account for filtering
start time sometimes randomly not zero: used ffmpeg to determine it instead of cv2 (may not work properly on macos-latest)
solved wrong padding of coordinates when person disappears from frame

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.5.2...v0.5.3

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Handle missing frames + config reorganization

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.5.1...v0.5.2

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Fixing edge cases

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.5.0...v0.5.1

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Get coordinates in meters

Get coordinates in meters
Load pose from trc instead of running inference
If multiperson=false, save person with highest average score over the whole sequence rather than just the first one

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.4.9...v0.5.0

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Option not to display and compute angles

calculate_angle = False

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.4.8...v0.4.9

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Fix UnboundLocalError in setup_webcam when not saving video

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.4.6...v0.4.8

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Fixed time range issue in command line

Full Changelog: https://github.com/davidpagnon/Sports2D/compare/v0.4.4...v0.4.6

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Case insensitive angles

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Various fixes + JOSS

Various fixes: - on webcam processing - angle plotting - issues on ubuntu - added a fontSize parameter

Edited JOSS paper:

Summary:

Sports2D provides a user-friendly solution for automatic and real-time analysis of multi-person human movement from a video or a webcam.

Statement of needs:

A large part of these tools focus on 2D analysis, such as OpenPose [@Cao2019], BlazePose [@Bazarevsky2020], or DeepLabCut [@Mathis2018]. More recently, RTMPose [@Jiang2023] offered a faster, more accurate, and more flexible alternative to the previous solutions.

Added method details

Sports2D: 1. Reads stream from a webcam, from one video, or from a list of videos. Selects the specified time range to process. 2. Sets up the RTMLib pose tracker from RTMlib with specified parameters. It can be run in lightweight, balanced, or performance mode, and for faster inference, keypoints can be tracked for a certain number of frames instead of detected. Any RTMPose model can be used. 3. Tracks people so that their IDs are consistent across frames. A person is associated to another in the next frame when they are at a small distance. IDs remain consistent even if the person disappears from a few frames. This carefully crafted sports2d tracker runs at a comparable speed as the RTMlib one but is much more robust. The user can still choose the RTMLib method if they need it by specifying it in the Config.toml file. . 4. Retrieves the keypoints with high enough confidence, and only keeps the persons with enough average high-confidence. 5. Computes the selected joint and segment angles, and flips them on the left/right side if the respective foot is pointing to the left/right. The user can select which angles they want to compute, display, and save. 5. Draws bounding boxes around each person and writes their IDs\ Draws the skeleton and the keypoints, with a green to red color scale to account for their confidence\ Draws joint and segment angles on the body, and writes the values either near the joint/segment, or on the upper-left of the image with a progress bar 6. Interpolates missing pose and angle sequences if gaps are not too large. Filters them with the selected filter (among Butterworth, Gaussian, LOESS, or Median) and their parameters 7. Optionally show processed images, saves them, or saves them as a video\ Optionally plots pose and angle data before and after processing for comparison\ Optionally saves poses for each person as a TRC file, and angles as a MOT file

The Demo video Sports2D is tested on is voluntarily challenging, in order to demonstrate the robustness of the process after sorting, interpolation and filtering. It contains: - One person walking in the sagittal plane - One person in the frontal plane. This person then performs a flip while being backlit, both of which are challenging for the pose detection algorithm - One tiny person flickering in the background who needs to be ignored

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Sports2D - Complete rewriting of the code

Rewrote the code after @hunminkim98's tests on (1) angle visualization, (2) with RTMPose, (3) including on webcam.

Gave an impulse to implement many other changes: - Used RTMPose rather than OpenPose or BlazePose (faster, more accurate, more flexible) - Single process for both pose and angle estimation - Webcam analysis available - Body angle visualization - Used entry points by default for clarity and simplicity - Saved as OpenSim files: TRC for pose, MOT for angles - Clearer documentation - Better demo video - Improved tracking IDs across frames - Added some segment angles: Pelvis, Shoulders, Head

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon over 1 year ago

Scientific Software - Peer-reviewed - Jupyter Notebook
Published by davidpagnon about 2 years ago

Recent Releases of Sports2D

Sports2D - Fixed GCV filtering + Expired OpenMMLab certificate

Sports2D - More filtering options (based on Pose2Sim)

Sports2D - trim video around valid frames

Sports2D - visible side (auto or manual) was not taken into account when flip_left_side = False

Sports2D - Various small fixes

Sports2D - stop freezing person at beginning and end of clips

Sports2D - v0.8.12

Sports2D - Support plotting on macOS

What's Changed

Sports2D - use_simple_model argument (faster but no muscles or flexible spine)

Sports2D - Little fixes for multi-threading and others

- Supported CUSTOM pose model

What's Changed

New Contributors

Sports2D - added feet_on_floor parameter and fixed whole_body pose estimation

Sports2D - Fixed 'on_click' ordering method

Sports2D - Small fix for new Pose2Sim version

Sports2D - v0.8.4

Sports2D - does not store all frames in memory if they are not saved or used for ordering

Sports2D - deepcopy of pose_model to make sure keypoint_ids are not overwritten

Sports2D - Fixed marker augmentation

Sports2D - Click and select the persons to analyze

Sports2D - Other minor edits

Sports2D - Minor improvements

What's Changed

Sports2D - Supported marker augmentation and inverse kinematics + others

What's Changed

Sports2D - c3d export + actual 3D trc files

Sports2D - DeepSort tracking and better skeleton visualization

Sports2D - Any detection and pose model can be used + more

Sports2D - v0.5.6

Sports2D - Important improvements

Sports2D - v0.5.4

Sports2D - Other edge cases fixed

Sports2D - Handle missing frames + config reorganization

Sports2D - Fixing edge cases

Sports2D - Get coordinates in meters

Sports2D - Option not to display and compute angles

Sports2D - Fix UnboundLocalError in setup_webcam when not saving video

Sports2D - Fixed time range issue in command line

Sports2D - Case insensitive angles

Sports2D - Various fixes + JOSS

Summary:

Statement of needs:

Added method details

Sports2D - Complete rewriting of the code

Sports2D - Better angle unwrapping and various fixes

Sports2D - fixed body_25 index error

Sports2D - Batch analysis enabled

Sports2D - Mixed up frames in video fixed

Sports2D - Colab version works again

Sports2D - fix comparison no longer supported by numpy

Sports2D - Small fix on Colab advanced params

Sports2D - clearer config parameters

Sports2D - Minor bug with certain associations of parameters

Sports2D - Colab set-up for demo

Sports2D - Added time in csv

Sports2D - New user friendly Colab version

Sports2D - Add option to save results to specific folder

Sports2D - All settings of pose and angle detection tested and work

Sports2D - Made it work in Colab

Sports2D - Correct requirement for python version

Sports2D - Publish on Pypi

Sports2D - First release