Data

Tools

Indexes

Applications

Experiments

Open Source Science

depyf

depyf is a tool to help you understand and adapt to PyTorch compiler torch.compile.

https://github.com/thuml/depyf

Science Score: 52.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ✓
    Institutional organization owner
    Organization thuml has institutional domain (ise.thss.tsinghua.edu.cn)
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.8%) to scientific vocabulary

Keywords

compiler deep-learning pytorch
Last synced: 6 months ago · JSON representation ·

Repository

depyf is a tool to help you understand and adapt to PyTorch compiler torch.compile.

Basic Info
Statistics
  • Stars: 718
  • Watchers: 9
  • Forks: 27
  • Open Issues: 1
  • Releases: 16
Topics
compiler deep-learning pytorch
Created over 2 years ago · Last pushed 10 months ago
Metadata Files
Readme License Citation

README.md

Logo

Documentation Status Supported Python Versions Python Decompilation Tests PyTorch Integration Tests Test Coverage MIT License

depyf is proud to be a PyTorch ecosystem project. Check out the announcement blog https://pytorch.org/blog/introducing-depyf/ for more details.

Have you ever felt overwhelmed by the complexities of torch.compile? Diving into its workings can feel like black magic, with bytecode and Python internal details that many users fail to understand, hindering them from understanding and adapting to torch.compile.

If you also face the problem, then you might be interested in depyf. As the logo suggests, depyf is a software tool to leverage advanced Python features (the Python snake symbol) to open up internal details (the internal gears symbol) of PyTorch's compiler torch.compile (the PyTorch logo), so that users can understand it, adapt to it, and tune their code (the debugger symbol) to get maximum performance benefit out of it.

:warning: This project is developed under close collaborations with the PyTorch team. Therefore, it requires very new features from PyTorch to support better understanding of torch.compile. Please use this project along with PyTorch>=2.2.0 (PyTorch nightly is recommended). Visit the PyTorch website for how to install different versions of PyTorch.

:warning: During development, we seek suggestions from the community quite a lot. You may find some early usage examples from some discussion forums or social media platforms. Please follow the latest documentation for how to use this tool.

Why depyf?

If you want to understand bytecode generated by torch.compile, then depyf might be the only choice for you. Below we tested several existing decompilers, they struggle to decompile simple Python bytecode across versions, and have poor support for PyTorch.

| Decompiler | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 | PyTorch | |-------------|--------------|------------|-------------|-------------|---------| | decompyle3 | 90.6% (77/85) | × | × | × | × | | uncompyle6 | 91.8% (78/85)| × | × | × | × | | pycdc | 74.1% (63/85) | 74.1% (63/85)| 74.1% (63/85) | 67.1% (57/85) | 19.3% (27/140)| | depyf | 100% (85/85) | 100% (85/85)| 100% (85/85)| 100% (85/85)| 100% (140/140)|

Installation

Stable release: pip install depyf

Nightly version (recommended): pip install git+https://github.com/thuml/depyf.git

Usage

The main usage is quite simple: just wrap your code within a context manager:

```python import torch from torch import _dynamo as torchdynamo from typing import List

@torch.compile def toy_example(a, b): x = a / (torch.abs(a) + 1) if b.sum() < 0: b = b * -1 return x * b

def main(): for _ in range(100): toy_example(torch.randn(10), torch.randn(10))

if name == "main": # main() # surround the code you want to run inside with depyf.prepare_debug import depyf with depyf.preparedebug("./dumpsrc_dir"): main() ```

Then you can see all the details of torch.compile inside the directory ./dump_src_dir. The details are organized into the following:

  • full_code_for_xxx.py for each function using torch.compile
  • __transformed_code_for_xxx.py for Python code associated with each graph.
  • __transformed_code_for_xxx.py.xxx_bytecode for Python bytecode, dumped code object, can be loaded via dill.load(open("/path/to/file", "wb")). Note that the load function might import some modules like transformers. Make sure you have these modules installed.
  • __compiled_fn_xxx.py for each computation graph and its optimization:
    • Captured Graph: a plain forward computation graph
    • Joint Graph: joint forward-backward graph from AOTAutograd
    • Forward Graph: forward graph from AOTAutograd
    • Backward Graph: backward graph from AOTAutograd
    • kernel xxx: compiled CPU/GPU kernel wrapper from Inductor.

We collect all the compilation artifacts when testing over 100 deep learning models. You can take a look to learn how the PyTorch compiler works.

If you want to use debugger to step through the above code, just add another context manager (and launch the script through debuggers):

```python import torch from torch import _dynamo as torchdynamo from typing import List

@torch.compile def toy_example(a, b): x = a / (torch.abs(a) + 1) if b.sum() < 0: b = b * -1 return x * b

def main(): for _ in range(100): toy_example(torch.randn(10), torch.randn(10))

if name == "main": import depyf with depyf.preparedebug("./dumpsrc_dir"): main() # surround the code you want to debug inside with depyf.debug() with depyf.debug(): main() ```

Calling depyf.debug() will pause the program for you to set breakpoints, and then you can use debuggers to hit breakpoints in these files under the ./dump_src_dir directory you specified above.

Contact

If you have any question about depyf, feel free to open issues to reach out! Any discussion/issue report/PR is welcome. Or contact youkaichao@gmail.com if you have any other questions.

Citing depyf

If you find depyf useful, please cite it in your publications.

latex @article{you2024depyf, title={depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers}, author={Kaichao You and Runsheng Bai and Meng Cao and Jianmin Wang and Ion Stoica and Mingsheng Long}, year={2024}, eprint={2403.13839}, journal={arXiv}, primaryClass={cs.LG}, url={https://github.com/thuml/depyf} }

Owner

  • Name: THUML @ Tsinghua University
  • Login: thuml
  • Kind: organization
  • Email: mingsheng@tsinghua.edu.cn

Machine Learning Group, School of Software, Tsinghua University

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "depyf"
  given-names: "contributors"
title: "depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers"
url: "https://github.com/thuml/depyf"
preferred-citation:
  type: article
  authors:
  - family-names: "You"
    given-names: "Kaichao"
  - family-names: "Bai"
    given-names: "Runsheng"
  - family-names: "Cao"
    given-names: "Meng"
  - family-names: "Wang"
    given-names: "Jianmin"
  - family-names: "Stoica"
    given-names: "Ion"
  - family-names: "Long"
    given-names: "Mingsheng"
  journal: "arxiv"
  # month: 9
  # start: 1 # First page number
  # end: 10 # Last page number
  title: "depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers"
  # issue: 1
  # volume: 1
  year: 2024

GitHub Events

Total
  • Create event: 8
  • Release event: 2
  • Issues event: 20
  • Watch event: 228
  • Delete event: 6
  • Issue comment event: 38
  • Push event: 25
  • Pull request review event: 5
  • Pull request event: 25
  • Fork event: 14
Last Year
  • Create event: 8
  • Release event: 2
  • Issues event: 20
  • Watch event: 228
  • Delete event: 6
  • Issue comment event: 38
  • Push event: 25
  • Pull request review event: 5
  • Pull request event: 25
  • Fork event: 14

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 34
  • Total pull requests: 49
  • Average time to close issues: 5 days
  • Average time to close pull requests: 2 days
  • Total issue authors: 24
  • Total pull request authors: 9
  • Average comments per issue: 2.71
  • Average comments per pull request: 0.76
  • Merged pull requests: 46
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 10
  • Pull requests: 21
  • Average time to close issues: 7 days
  • Average time to close pull requests: 1 day
  • Issue authors: 10
  • Pull request authors: 6
  • Average comments per issue: 1.8
  • Average comments per pull request: 1.24
  • Merged pull requests: 18
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • tombousso (5)
  • imShZh (4)
  • xmfan (3)
  • youkaichao (2)
  • jeromeku (2)
  • emmyxth (1)
  • tbaggu (1)
  • vimarsh6739 (1)
  • xwu99 (1)
  • jon-chuang (1)
  • s1mplezpc (1)
  • gilfree (1)
  • willdla (1)
  • xgbj (1)
  • nopperl (1)
Pull Request Authors
  • youkaichao (69)
  • zincnode (4)
  • tombousso (4)
  • imShZh (3)
  • ProExpertProg (2)
  • nopperl (2)
  • Brs-sss (2)
  • bjhargrave (2)
  • gilfree (2)
Top Labels
Issue Labels
bug (13) usage (2)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 2,485,722 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 1
  • Total versions: 20
  • Total maintainers: 1
pypi.org: depyf

Decompile python functions, from bytecode to source code!

  • Versions: 20
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 2,485,722 Last month
Rankings
Stargazers count: 5.9%
Dependent packages count: 10.1%
Downloads: 13.4%
Average: 13.6%
Forks count: 16.9%
Dependent repos count: 21.5%
Maintainers (1)
youkaichao
Last synced: 6 months ago

Dependencies

setup.py pypi