Data

Tools

Indexes

Applications

Experiments

Open Source Science

viztracer

A debugging and profiling tool that can trace and visualize python code execution

https://github.com/gaogaotiantian/viztracer

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.6%) to scientific vocabulary

Keywords

debugging flamegraph logging profiling python python3 tracer visualization

Keywords from Contributors

closember jax
Last synced: 4 months ago · JSON representation ·

Repository

A debugging and profiling tool that can trace and visualize python code execution

Basic Info
Statistics
  • Stars: 6,967
  • Watchers: 58
  • Forks: 446
  • Open Issues: 23
  • Releases: 88
Topics
debugging flamegraph logging profiling python python3 tracer visualization
Created over 5 years ago · Last pushed 7 months ago
Metadata Files
Readme Contributing Funding License Citation

README.md

VizTracer

build flake8 readthedocs coverage pypi Visual Studio Marketplace Version support-version license commit sponsor

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

The front-end UI is powered by Perfetto. Use "AWSD" to zoom/navigate. More help can be found in "Support - Controls".

example_img

Highlights

  • Detailed function entry/exit information on timeline with source code
  • Super easy to use, no source code change for most features, no package dependency
  • Low overhead, probably the fastest tracer in the market
  • Supports threading, multiprocessing, subprocess, async and PyTorch
  • Powerful front-end, able to render GB-level trace smoothly
  • Works on Linux/MacOS/Windows

Install

The preferred way to install VizTracer is via pip

sh pip install viztracer

Basic Usage

Command Line

```sh

Instead of "python3 my_script.py arg1 arg2"

viztracer my_script.py arg1 arg2 ```

A result.json file will be generated, which you can open with vizviewer ```sh # You can display all the files in a directory and open them in browser too vizviewer ./ # For very large trace files, try external trace processor vizviewer --use_external_processor result.json ``` vizviewer will host an HTTP server on ``http://localhost:9001``. You can also open your browser and use that address. If you do not want vizviewer to open the webbrowser automatically, you can use ```sh vizviewer --server_only result.json ``` If you just need to bring up the trace report once, and do not want the persistent server, use ```sh vizviewer --once result.json ```

sh vizviewer result.json

A VS Code Extension is available to make your life even easier.

Add --open to open the reports right after tracing ```sh viztracer --open my_script.py arg1 arg2 viztracer -o result.html --open my_script.py arg1 arg2 ```
modules and console scripts(like flask) are supported as well ```sh viztracer -m your_module ``` ```sh viztracer flask run ```

Inline

You can also manually start/stop VizTracer in your script as well.

```python from viztracer import VizTracer

tracer = VizTracer() tracer.start()

Something happens here

tracer.stop() tracer.save() # also takes output_file as an optional argument ```

Or, you can do it with with statement

python with VizTracer(output_file="optional.json") as tracer: # Something happens here

Jupyter

If you are using Jupyter, you can use viztracer cell magics.

```python

You need to load the extension first

%load_ext viztracer ```

```python %%viztracer

Your code after

```

A VizTracer Report button will appear after the cell and you can click it to view the results

PyTorch

VizTracer can log native calls and GPU events of PyTorch (based on torch.profiler) with --log_torch.

python with VizTracer(log_torch=True) as tracer: # Your torch code

sh viztracer --log_torch your_model.py

Advanced Usage

Trace Filter

VizTracer can filter out the data you don't want to reduce overhead and keep info of a longer time period before you dump the log.

Extra Logs without Code Change

VizTracer can log extra information without changing your source code

Add Custom Event

VizTracer supports inserting custom events while the program is running. This works like a print debug, but you can know when this print happens while looking at trace data.

Misc

Multi Thread Support

For Python3.12+, VizTracer supports Python-level multi-thread tracing without the need to do any modification to your code.

For versions before 3.12, VizTracer supports python native threading module. Just start VizTracer before you create threads and it will just work.

For other multi-thread scenarios, you can use enable_thread_tracing() to notice VizTracer about the thread to trace it.

example_img

Refer to multi thread docs for details

Multi Process Support

VizTracer supports subprocess, multiprocessing, os.fork(), concurrent.futures, and loky out of the box.

For more general multi-process cases, VizTracer can support with some extra steps.

example_img

Refer to multi process docs for details

Async Support

VizTracer supports asyncio natively, but could enhance the report by using --log_async.

example_img

Refer to async docs for details

Flamegraph

Perfetto supports native flamegraph, just select slices on the UI and choose "Slice Flamegraph".

example_img

Remote attach

VizTracer supports remote attach to an arbitrary Python process to trace it, as long as viztracer is importable

Refer to remote attach docs

JSON alternative

VizTracer needs to dump the internal data to json format. It is recommended for the users to install orjson, which is much faster than the builtin json library. VizTracer will try to import orjson and fall back to the builtin json library if orjson does not exist.

Performance

VizTracer puts in a lot of effort to achieve low overhead. The actual performance impact largely depends on your application. For typical codebases, the overhead is expected to be below 1x. If your code has infrequent function calls, the overhead could be minimal.

Detailed explanation The overhead introduced by VizTracer is basically a fixed amount of time during function entry and exit, so the more time spent on function entries and exits, the more overhead will be observed. A pure recursive ```fib``` function could suffer 3x-4x overhead on Python3.11+ (when the Python call is optimized, before that Python call was slower so the overhead ratio would be less). In the real life scenario, your code should not spend too much time on function calls (they don't really do anything useful), so the overhead would be much smaller. Many techniques are applied to minimize the overall overhead during code execution to reduce the inevitable skew introduced by VizTracer (the report saving part is not as critical). For example, VizTracer tries to use the CPU timestamp counter instead of a syscall to get the time when available. On Python 3.12+, VizTracer uses ```sys.monitoring``` which has less overhead than ```sys.setprofile```. All of the efforts made it observably faster than ```cProfile```, the Python stdlib profiler. However, VizTracer is a tracer, which means it has to record every single function entry and exit, so it can't be as fast as the sampling profilers - they are not the same thing. With the extra overhead, VizTracer provides a lot more information than normal sampling profilers.

Documentation

For full documentation, please see https://viztracer.readthedocs.io/en/stable

Bugs/Requests

Please send bug reports and feature requests through github issue tracker. VizTracer is currently under development now and it's open to any constructive suggestions.

License

Copyright 2020-2025 Tian Gao.

Distributed under the terms of the Apache 2.0 license.

Owner

  • Login: gaogaotiantian
  • Kind: user
  • Location: Irvine
  • Company: Microsoft

Author of VizTracer, watchpoints and objprint

Citation (CITATION.cff) 

cff-version: 1.2.0
title: >-
  VizTracer - A debugging and profiling tool that can trace
  and visualize python code execution
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Tian
    family-names: Gao
    email: gaogaotiantian@hotmail.com
    orcid: 'https://orcid.org/0009-0006-9702-7067'
repository-code: 'https://github.com/gaogaotiantian/viztracer'
license: Apache-2.0

Committers

Last synced: 8 months ago

All Time
  • Total Commits: 685
  • Total Committers: 31
  • Avg Commits per committer: 22.097
  • Development Distribution Score (DDS): 0.083
Past Year
  • Commits: 92
  • Committers: 6
  • Avg Commits per committer: 15.333
  • Development Distribution Score (DDS): 0.065
Top Committers
Name Email Commits
gaogaotiantian g****n@h****m 628
Tianshun_Peng 3****n 16
谭九鼎 1****3@q****m 4
Sam Gross c****y@g****m 3
F-park 5****k 2
Sefank 1****k 2
Sergey Miryanov s****v@g****m 2
TAN Long 7****g 2
xzxcxzxcxzxc 8****c 2
yihong z****8@g****m 2
zyckk4 1****4 2
Alexander Zhipa a****a@p****e 1
AlexeyKozhevin a****n@g****m 1
Comzyh c****h 1
Conor O'Callaghan i****s 1
DancingSnow 6****7 1
David de la Iglesia Castro d****o@g****m 1
Kevin Kuriakose t****5@g****m 1
Kian-Meng Ang k****g@g****m 1
Maarten Breddels m****s@g****m 1
Malte Werschy w****m@g****m 1
Olivier Grisel o****l@e****g 1
Peter Stolz 5****z 1
Shawn Lin y****z@g****m 1
Yang Liu l****3@g****m 1
acehinnnqru a****u@g****m 1
luoos l****o@g****m 1
monkeywq 3****q 1
odidev o****v@p****m 1
richardsheridan r****n@g****m 1
and 1 more...
Committer Domains (Top 20 + Academic)
puresoftware.com: 1 ensta.org: 1 proton.me: 1 qq.com: 1

Issues and Pull Requests

Last synced: 4 months ago

All Time
  • Total issues: 156
  • Total pull requests: 322
  • Average time to close issues: 2 months
  • Average time to close pull requests: 4 days
  • Total issue authors: 120
  • Total pull request authors: 38
  • Average comments per issue: 2.78
  • Average comments per pull request: 1.7
  • Merged pull requests: 268
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 49
  • Pull requests: 178
  • Average time to close issues: 10 days
  • Average time to close pull requests: about 19 hours
  • Issue authors: 37
  • Pull request authors: 12
  • Average comments per issue: 1.47
  • Average comments per pull request: 1.19
  • Merged pull requests: 147
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • gaogaotiantian (15)
  • Chang-LeHung (5)
  • yihong0618 (3)
  • AlexeyKozhevin (3)
  • xpf629629 (3)
  • zplizzi (3)
  • gedalia (2)
  • arbenede (2)
  • pulkin (2)
  • FindDefinition (2)
  • RedSiamese (2)
  • kloczek (2)
  • Natural-selection1 (2)
  • seniorsolt (2)
  • auderson (2)
Pull Request Authors
  • gaogaotiantian (224)
  • TTianshun (20)
  • Chang-LeHung (10)
  • sergey-miryanov (4)
  • user-11150 (4)
  • HernandoR (4)
  • colesbury (4)
  • yihong0618 (4)
  • xzxcxzxcxzxc (3)
  • F-park (3)
  • JessyTsu1 (2)
  • pulkin (2)
  • Technohacker (2)
  • kursataktas (2)
  • yodahuang (2)
Top Labels
Issue Labels
bug (5) question (3) enhancement (2) deficiency (2) help wanted (2) unstable-test (2) need further investigation (2) not a bug (1) wontfix (1) good first issue (1)
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 634,589 last-month
  • Total docker downloads: 73
  • Total dependent packages: 6
    (may contain duplicates)
  • Total dependent repositories: 43
    (may contain duplicates)
  • Total versions: 103
  • Total maintainers: 1
pypi.org: viztracer

A debugging and profiling tool that can trace and visualize python code execution

  • Homepage: https://github.com/gaogaotiantian/viztracer
  • Documentation: https://viztracer.readthedocs.io
  • License: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ======================================================================== VizTracer Subcomponents: VizTracer contains subcomponents with separate copyright notices and license terms. Your use of the source code for these subcomponents is also subject to the terms and conditions of the following licenses. BSD-3-Clause (src/viztracer/html/LICENSE): src/viztracer/html/trace_viewer_*.html Apache License V2 (src/viztracer/web_dist/LICENSE): src/viztracer/web_dist/* Eclipse Public License v1.0 (src/viztracer/attach_process/LICENSE) src/viztracer/attach_process/*
  • Latest release: 1.0.4
    published 8 months ago
  • Versions: 84
  • Dependent Packages: 6
  • Dependent Repositories: 43
  • Downloads: 634,589 Last month
  • Docker Downloads: 73
Rankings
Stargazers count: 1.3%
Downloads: 1.8%
Dependent repos count: 2.2%
Dependent packages count: 2.4%
Average: 2.4%
Forks count: 2.8%
Docker downloads count: 3.8%
Maintainers (1)
gaogaotiantian
Last synced: 4 months ago
conda-forge.org: viztracer
  • Versions: 19
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Stargazers count: 7.2%
Forks count: 9.7%
Average: 25.5%
Dependent repos count: 34.0%
Dependent packages count: 51.2%
Last synced: 4 months ago

Dependencies

.github/workflows/docs.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/lint.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/python-package.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • codecov/codecov-action v3 composite
  • maxim-lobanov/setup-xcode v1 composite
.github/workflows/release.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • docker/setup-qemu-action v1 composite
.github/workflows/wheels.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • docker/setup-qemu-action v1 composite
docs/requirements.txt pypi
  • Sphinx ==7.2.6
  • sphinx-rtd-theme ==1.3.0
example/requirements.txt pypi
  • mcts *
  • numpy *
pyproject.toml pypi
  • objprint >0.1.3
requirements-dev.txt pypi
  • build * development
  • coverage * development
  • flake8 * development
  • ipywidgets >8.0.5 development
  • jaxlib * development
  • loky >=3.0.0 development
  • mypy * development
  • setuptools * development
  • twine * development
  • wheel * development
setup.py pypi