Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/bonsai-rx/deeplabcut

A Bonsai interface for real-time markerless pose estimation using DeepLabCut

https://github.com/bonsai-rx/deeplabcut

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 3 committers (33.3%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.4%) to scientific vocabulary

Keywords

bonsai-rx deeplabcut tensorflow
Last synced: 5 months ago · JSON representation

Repository

A Bonsai interface for real-time markerless pose estimation using DeepLabCut

Basic Info
Statistics
  • Stars: 51
  • Watchers: 10
  • Forks: 10
  • Open Issues: 10
  • Releases: 4
Topics
bonsai-rx deeplabcut tensorflow
Created almost 6 years ago · Last pushed 9 months ago
Metadata Files
Readme License

docs/README.md

Bonsai - DeepLabCut

logo

Bonsai.DeepLabCut is a Bonsai interface for DeepLabCut allowing real-time markerless pose estimation using pretrained models stored in the Protocol Buffers (pb) format. Natively, DLC stores the result of training as checkpoints, but it is possible to export them to .pb files using the deeplabcut.export_model function within DLC (Read more here).

Bonsai.DeepLabCut loads these .pb files using TensorFlowSharp, a set of .NET bindings for TensorFlow allowing native inference using either the CPU or GPU. By using the .pb file and the pose configuration YAML (pose_cfg.yaml), the DetectPose operator from Bonsai.DeepLabCut automatically sets up an inference graph and feeds it with live image data coming from any other Bonsai image source. The output is a Pose class which you can access to extract specific body parts, filter out invalid positions using a confidence threshold, or record using CsvWriter.

The Bonsai.DeepLabCut project was kickstarted at the DeepLabCut Hackathon sponsored by the Chan Zuckerberg Initiative and held at Harvard University in March 2020. It has since been published in eLife (Kane et al, eLife 2020).

How to install

Bonsai.DeepLabCut can be downloaded through the Bonsai package manager. In order to get visualizer support, you should download both the Bonsai.DeepLabCut and Bonsai.DeepLabCut.Design packages. However, in order to use it for either CPU or GPU inference, you need to pair it with a compiled native TensorFlow binary. You can find precompiled binaries for Windows 64-bit at https://www.tensorflow.org/install/lang_c.

To use GPU TensorFlow (highly recommended for live inference), you also need to install the CUDA Toolkit 11.2 from the CUDA Toolkit Archive, and download cuDNN 8.1.0 for CUDA 11.2. Make sure you have a CUDA 11 compatible GPU with the latest NVIDIA drivers.

After downloading the native TensorFlow binary and cuDNN, you can follow these steps to get the required native files into the Extensions folder of your local Bonsai install:

  1. The easiest way to find your Bonsai install folder is to right-click on the Bonsai shortcut > Properties. The path to the folder will be shown in the "Start in" textbox;
  2. Copy tensorflow.dll file from either the CPU or GPU tensorflow release to the Extensions folder;
  3. If you are using TensorFlow GPU, make sure to add the cuda/bin folder of your cuDNN download to the PATH environment variable, or copy all DLL files to the Extensions folder.

How to use

The core operator of Bonsai.DeepLabCut is the DetectPose node. You can place it after any image source, like so:

detect-pose.svg

You will also need to point the ModelFileName to the exported .pb file containing your pretrained DLC model, and the PoseConfigFileName to the pose_cfg.yaml file describing the joint labels of the pose skeleton.

If everything works out, you should see some indications in the Bonsai command line window about whether the GPU was successfully detected and enabled. The first frame will cold start the inference graph which may take a bit of time, but after that your poses should start streaming through!

Short DLC install guide

For all questions regarding installation of DeepLabCut, please check the official docs. However, we did find the following build steps to be reliable for a self-contained barebones install on Windows at the time of writing:

  1. Download WinPython 3.8.10 64bit only from GitHub.
  2. Extract to a folder and launch WinPython Command Prompt.exe.
  3. pip install tensorflow==2.8.0 or pip install tensorflow-gpu==2.8.0 depending on whether you will be using CPU or GPU TensorFlow.
  4. pip install deeplabcut[gui]==2.2.0.6.
  5. Launch the DLC gui: python -m deeplabcut.

Owner

  • Name: Bonsai
  • Login: bonsai-rx
  • Kind: organization

A visual language for reactive programming

GitHub Events

Total
  • Issues event: 2
  • Watch event: 3
  • Delete event: 3
  • Push event: 2
  • Pull request review event: 1
  • Pull request event: 2
  • Fork event: 1
  • Create event: 3
Last Year
  • Issues event: 2
  • Watch event: 3
  • Delete event: 3
  • Push event: 2
  • Pull request review event: 1
  • Pull request event: 2
  • Fork event: 1
  • Create event: 3

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 46
  • Total Committers: 3
  • Avg Commits per committer: 15.333
  • Development Distribution Score (DDS): 0.087
Past Year
  • Commits: 4
  • Committers: 2
  • Avg Commits per committer: 2.0
  • Development Distribution Score (DDS): 0.25
Top Committers
Name Email Commits
glopesdev g****v 42
David Maas d****t@n****g 3
Mackenzie Mathis m****s@r****u 1
Committer Domains (Top 20 + Academic)
rowland.harvard.edu: 1 neurogears.org: 1

Issues and Pull Requests

Last synced: 7 months ago

All Time
  • Total issues: 28
  • Total pull requests: 2
  • Average time to close issues: 4 months
  • Average time to close pull requests: 7 days
  • Total issue authors: 17
  • Total pull request authors: 2
  • Average comments per issue: 1.25
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 1
  • Average time to close issues: 2 months
  • Average time to close pull requests: 14 days
  • Issue authors: 1
  • Pull request authors: 1
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • glopesdev (5)
  • chiyu1203 (2)
  • caniko (2)
  • emilyjanedennis (1)
  • banchan86 (1)
  • JensBlack (1)
  • GabrielBonassi77 (1)
  • kvnlxm (1)
  • DaBue93 (1)
  • horsto (1)
  • T0mOmine (1)
  • rcbullins (1)
  • mrizaarseven98 (1)
  • MilanNedoma (1)
  • tokioshima (1)
Pull Request Authors
  • PathogenDavid (2)
  • MMathisLab (1)
Top Labels
Issue Labels
proposal (2) feature (2) bug (1) question (1) documentation (1)
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • nuget 8,553 total
  • Total dependent packages: 1
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 8
  • Total maintainers: 2
nuget.org: bonsai.deeplabcut

Bonsai Library for real-time markerless pose estimation using DeepLabCut. TensorFlow 2.8.0 or compatible is required.

  • Versions: 4
  • Dependent Packages: 1
  • Dependent Repositories: 0
  • Downloads: 4,454 Total
Rankings
Dependent repos count: 10.2%
Dependent packages count: 13.9%
Average: 17.6%
Downloads: 28.7%
Maintainers (2)
glopes bonsai
Last synced: 6 months ago
nuget.org: bonsai.deeplabcut.design

Bonsai Library containing reactive visualizations for markerless pose estimation using DeepLabCut.

  • Versions: 4
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 4,099 Total
Rankings
Dependent repos count: 10.2%
Dependent packages count: 13.9%
Average: 18.0%
Downloads: 29.8%
Maintainers (2)
glopes bonsai
Last synced: 6 months ago