Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/awslabs/mxboard

Logging MXNet data for visualization in TensorBoard.

https://github.com/awslabs/mxboard

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 9 committers (11.1%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.0%) to scientific vocabulary

Keywords from Contributors

mxnet
Last synced: 11 months ago · JSON representation

Repository

Logging MXNet data for visualization in TensorBoard.

Basic Info
  • Host: GitHub
  • Owner: awslabs
  • License: apache-2.0
  • Language: Python
  • Default Branch: master
  • Size: 108 KB
Statistics
  • Stars: 325
  • Watchers: 24
  • Forks: 47
  • Open Issues: 15
  • Releases: 0
Archived
Created over 8 years ago · Last pushed over 4 years ago
Metadata Files
Readme Contributing License Code of conduct

README.md

Logging MXNet Data for Visualization in TensorBoard

Overview

MXBoard provides a set of APIs for logging MXNet data for visualization in TensorBoard. The idea of this project comes from discussions with Zihao Zheng, the author of dmlc/tensorboard, on delivering a visualization solution for MXNet users. We aim at providing the logging APIs that can process MXNet data efficiently and supporting most of the data types for visualization in the TensorBoard GUI. We adapted the following low-level logging components from their Python and C++ implementations in TensorFlow: FileWriter, EventFileWriter, EventsWriter, RecordWriter, and _EventLoggerThread. We also adapted the user-level logging APIs defined in SummaryWriter from tensorboard-pytorch. The encoding algorithm used in writing protobuf objects into event files is directly borrowed from TeamHG-Memex/tensorboard_logger.

MXBoard supports a set of Python APIs for logging the following data types for TensorBoard to render. Logging APIs for other languages may be added in the future.

mxboard_cover

The corresponding Python APIs are accessible through a class called SummaryWriter as follows:

python mxboard.SummaryWriter.add_graph mxboard.SummaryWriter.add_scalar mxboard.SummaryWriter.add_histogram mxboard.SummaryWriter.add_embedding mxboard.SummaryWriter.add_image mxboard.SummaryWriter.add_text mxboard.SummaryWriter.add_pr_curve mxboard.SummaryWriter.add_audio

Installation

Install MXBoard from PyPI

bash pip install mxboard

Install MXBoard Python package from source

bash git clone https://github.com/awslabs/mxboard.git cd mxboard/python python setup.py install

Install TensorBoard from PyPI

MXBoard is a logger for writing MXNet data to event files. To visualize those data in browsers, users still have to install TensorBoard separately.

bash pip install tensorboard

Use the following to verify that the TensorBoard binary has been installed correctly.

bash tensorboard --help

Other required packages

MXBoard relies on the following packages for data logging. - MXNet - protobuf3 - six - Pillow

Please note that you need to install MXNet manually before using MXBoard. The other packages will be installed automatically when you install MXBoard via pip or building from source. If you want to build from source, please make sure that protobuf compiler is installed. Check this page for downloading the protobuf compiler whose file name starts with "protoc".

Visualizing MXNet data in 30 seconds

Now that you have installed all of the required packages, let's walk through a simple visualization example. You will see how MXBoard enables visualizing MXNet NDArrays with histograms.

Step 1. Logging event data to a file.

Prepare a Python script for writing data generated by the normal operator to an event file. The data is generated ten times with decreasing standard deviation and written to the event file each time. It's expected to see the data distribution gradually become more centered around the mean value. Note that here we specify creating the event file in the folder logs under the current directory. We will need to pass this folder path to the TensorBoard binary. ```python import mxnet as mx from mxboard import SummaryWriter

with SummaryWriter(logdir='./logs') as sw: for i in range(10): # create a normal distribution with fixed mean and decreasing std data = mx.nd.normal(loc=0, scale=10.0/(i+1), shape=(10, 3, 8, 8)) sw.addhistogram(tag='normldist', values=data, bins=200, global_step=i) ```

Step 2. Launch TensorBoard to load the event file generated above.

Use the following command to start the TensorBoard server. It will use the logs that were generated in the current directory's logs folder. bash tensorboard --logdir=./logs --host=127.0.0.1 --port=8888 Note that in some situations, the port number 8888 may be occupied by other applications and launching TensorBoard may fail. You may choose a different available port number.

Step 3. Open TensorBoard in your browser.

In the browser, enter the address 127.0.0.1:8888, and click the tab HISTOGRAMS in the TensorBoard GUI. You will see data distribution changing as time progresses.

summary_histogram_norm

More tutorials

References

  1. https://github.com/TeamHG-Memex/tensorboard_logger
  2. https://github.com/lanpa/tensorboard-pytorch
  3. https://github.com/dmlc/tensorboard
  4. https://github.com/tensorflow/tensorflow
  5. https://github.com/tensorflow/tensorboard

License

This library is licensed under the Apache 2.0 License.

Owner

  • Name: Amazon Web Services - Labs
  • Login: awslabs
  • Kind: organization
  • Location: Seattle, WA

AWS Labs

GitHub Events

Total
  • Watch event: 3
Last Year
  • Watch event: 3

Committers

Last synced: over 3 years ago

All Time
  • Total Commits: 42
  • Total Committers: 9
  • Avg Commits per committer: 4.667
  • Development Distribution Score (DDS): 0.333
Top Committers
Name Email Commits
reminisce w****u@g****m 28
Jacob Peddicord j****d@u****m 7
Yong Sun m****l@y****e 1
Samantha fischer 4****r@u****m 1
YusongLeng 4****g@u****m 1
Jesse Brizzi j****i@u****m 1
acmore h****b@g****m 1
Philip Hyunsu Cho c****1@c****u 1
Aaron Markham m****a@a****m 1
Committer Domains (Top 20 + Academic)
amazon.com: 1 cs.washington.edu: 1 yongsun.me: 1

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 31
  • Total pull requests: 19
  • Average time to close issues: 8 months
  • Average time to close pull requests: 11 days
  • Total issue authors: 25
  • Total pull request authors: 9
  • Average comments per issue: 2.55
  • Average comments per pull request: 0.84
  • Merged pull requests: 18
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • fhieber (3)
  • reminisce (2)
  • hskramer (2)
  • jacky4323 (2)
  • leezu (2)
  • Ved111 (1)
  • Tyhye (1)
  • HarveyGuo960817 (1)
  • luoyetx (1)
  • ShownX (1)
  • casscw (1)
  • bkktimber (1)
  • LourisXu (1)
  • firestonelib (1)
  • lambdaofgod (1)
Pull Request Authors
  • reminisce (11)
  • acmore (1)
  • yongsun (1)
  • hcho3 (1)
  • aaronmarkham (1)
  • jessebrizzi (1)
  • YusongLeng (1)
  • SamanthaFeidFischer (1)
  • 372046933 (1)
Top Labels
Issue Labels
bug (4) question (4) Feature Request (4) doc (2) good first issue (1)
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 1,730 last-month
  • Total docker downloads: 49,350
  • Total dependent packages: 1
    (may contain duplicates)
  • Total dependent repositories: 113
    (may contain duplicates)
  • Total versions: 13
  • Total maintainers: 1
pypi.org: mxboard

A logging tool for visualizing MXNet data in TensorBoard

  • Versions: 12
  • Dependent Packages: 1
  • Dependent Repositories: 113
  • Downloads: 1,730 Last month
  • Docker Downloads: 49,350
Rankings
Docker downloads count: 1.0%
Dependent repos count: 1.4%
Average: 3.3%
Downloads: 3.5%
Stargazers count: 3.5%
Dependent packages count: 4.6%
Forks count: 5.8%
Maintainers (1)
aws-deep-learning
Last synced: 11 months ago
proxy.golang.org: github.com/awslabs/mxboard
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 7.0%
Average: 8.2%
Dependent repos count: 9.3%
Last synced: 11 months ago