dex-lang

Research language for array processing in the Haskell/ML family

https://github.com/google-research/dex-lang

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
    Links to: arxiv.org
  • Committers with academic emails
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (13.0%) to scientific vocabulary

Keywords from Contributors

jax
Last synced: 7 months ago · JSON representation ·

Repository

Research language for array processing in the Haskell/ML family

Basic Info
  • Host: GitHub
  • Owner: google-research
  • License: bsd-3-clause
  • Language: Haskell
  • Default Branch: main
  • Homepage:
  • Size: 34.5 MB
Statistics
  • Stars: 1,617
  • Watchers: 56
  • Forks: 110
  • Open Issues: 141
  • Releases: 0
Created over 6 years ago · Last pushed about 1 year ago
Metadata Files
Readme Contributing License Citation

README.md

Dex Test status

Dex (named for "index") is a research language for typed, functional array processing. The goal of the project is to explore:

  • Type systems for array programming
  • Mathematical program transformations like differentiation and integration
  • User-directed compilation to parallel hardware
  • Interactive and incremental numerical programming and visualization

To learn more, check out our paper, our tutorial or these example programs:

Or for a more comprehensive look, there's

  • The InDex of all documents, libraries, and examples included in this repository.

🚨 Dex is an experimental research project at an early stage of development. Expect monstrous bugs and razor-sharp edges!

🤝 Contributions welcome! See our issue tracker for good first issues, or browse by thematic labels.

Dependencies

  • Install stack
  • Install LLVM 12
    • Ubuntu/Debian: apt-get install llvm-12-dev
    • macOS: brew install llvm@12
      • Make sure llvm@12 is on your PATH before building. Example: export PATH="$(brew --prefix llvm@12)/bin:$PATH"
  • Install clang 12 (may be installed together with llvm)
    • Ubuntu/Debian: apt-get install clang-12
    • macOS: installs with llvm
  • Install libpng (often included by default in *nix platforms)
    • Ubuntu/Debian: apt-get install libpng-dev
    • macOS: brew install libpng

Installing

To build and install a release version of Dex run make install.

The default installation directory is $HOME/.local/bin, so make sure to add that directory to $PATH after installing Dex. To install Dex somewhere else, set the PREFIX environment variable before running make install. For example, PREFIX=$HOME make install installs dex in $HOME/bin.

Building

  • Build Dex in development (unoptimized) mode: make
  • Run tests in development mode: make tests

It is convenient to set up a dex alias (e.g. in .bashrc) for running Dex in development mode:

```console

Linux:

alias dex="stack exec dex -- --lib-path lib"

macOS:

alias dex="stack exec --stack-yaml=stack-macos.yaml dex -- --lib-path lib" ```

You might also want to consider other development build targets: * build-opt for local optimized builds, * build-dbg for local debug builds, * build-prof for local optimized builds with profiling enabled.

Those non-standard targets require different aliases. Please consult the documentation at the top of the makefile for detailed instructions.

Haskell Language Server

In order to use HLS with the Haskell code in this project:

  • Install ghcup
  • Run ghcup install hls
  • Create a file in the root dex directory called hie.yaml with the following contents:

yaml cradle: stack: stackYaml: "./stack-macos.yaml" # Or stack.yaml if not on MacOS

Unfortunately one cannot dynamically select the stack.yaml file to use based on the environment, and so one has to create an appropriate hie.yaml file manually. This will be ignored by git.

This should work out of the box with Emacs' lsp-haskell package.

Building with Nix

Nix is a functional package manager and build system.

To build with vanilla Nix: bash $ nix-build

To build with flakes-enabled Nix: bash $ nix build .#dex The resulting dex binary should be in result/bin/dex.

For development purposes, you can use a Nix environment with bash $ nix-shell $ nix develop # With flakes and use make to use Stack to build Dex.

Running

  • Traditional REPL: dex repl
  • Execute script: dex script examples/pi.dx
  • Live-updated notebook display dex web examples/pi.dx (html) or dex watch examples/pi.dx (terminal).

License

BSD-3

This is an early-stage research project, not an official Google product.

Owner

  • Name: Google Research
  • Login: google-research
  • Kind: organization
  • Location: Earth

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- given-names: "Dougal"
  family-names: "Maclaurin"
- given-names: "Adam"
  family-names: "Paszke"
title: "Dex: typed and functional array processing"
version: 0.0.0
date-released: 2018-09-29
url: "https://github.com/google-research/dex-lang"
preferred-citation:
  type: article
  authors:
  - given-names: "Adam"
    family-names: "Paszke"
  - given-names: "Daniel D."
    family-names: "Johnson"
  - given-names: "David"
    family-names: "Duvenaud"
  - given-names: "Dimitrios"
    family-names: "Vytiniotis"
  - given-names: "Alexey"
    family-names: "Radul"
  - given-names: "Matthew J."
    family-names: "Johnson"
  - given-names: "Jonathan"
    family-names: "Ragan-Kelley"
  - given-names: "Dougal"
    family-names: "Maclaurin"
  doi: "10.1145/3473593"
  title: "Getting to the Point: Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming"
  volume: 5
  number: "ICFP"
  year: 2021
  month: 8
  issue-date: "August 2021"
  journal: "Proceedings of the ACM on Programming Languages"
  pages: 29
  abstract: >
    We present a novel programming language design that attempts to combine the clarity
    and safety of high-level functional languages with the efficiency and parallelism
    of low-level numerical languages. We treat arrays as eagerly-memoized functions on
    typed index sets, allowing abstract function manipulations, such as currying, to work
    on arrays. In contrast to composing primitive bulk-array operations, we argue for
    an explicit nested indexing style that mirrors application of functions to arguments.
    We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized
    in-place updates. Specifically, an associative accumulation effect allows reverse-mode
    automatic differentiation of in-place updates in a way that preserves parallelism.
    Empirically, we benchmark against the Futhark array programming language, and demonstrate
    that aggressive inlining and type-driven compilation allows array programs to be written
    in an expressive, "pointful" style with little performance penalty.
  keywords:
  - "array programming"
  - "automatic differentiation"
  - "parallel computing"
  url: "https://doi.org/10.1145/3473593"
  publisher:
    name: "Association for Computing Machinery"
    city: "New York"
    region: "NY"
    country: "USA"

GitHub Events

Total
  • Issues event: 1
  • Watch event: 51
  • Push event: 14
  • Fork event: 7
Last Year
  • Issues event: 1
  • Watch event: 51
  • Push event: 14
  • Fork event: 7

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 3,058
  • Total Committers: 42
  • Avg Commits per committer: 72.81
  • Development Distribution Score (DDS): 0.428
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Dougal Maclaurin d****m@g****m 1,748
Adam Paszke a****e@g****m 469
Alexey Radul a****h@g****m 390
David Duvenaud d****d@g****m 83
Daniel Johnson d****n@g****m 72
Lyndon White l****e@i****k 59
Dan Zheng d****g@g****m 43
srush s****h@g****m 29
Norman Rink n****k@g****m 22
Matthew Johnson m****j@g****m 18
chenzizhao z****o@g****m 12
Yonghao Jin y****1@g****m 11
joaogui1 j****o@g****m 11
Emily Fertig e****f@g****m 10
Darren Wilkinson d****n@b****m 10
Alex Reinking r****g@g****m 8
Niklas Schmitz n****z@g****m 7
DHHZ d****g@g****m 6
Jonathan Merritt j****t@g****m 4
Peter Becich p****h@g****m 4
Sharad Vikram s****m@g****m 4
Torsten Scholak t****k@g****m 4
CJ Carey p****e@g****m 4
Benjamin Chetioui 3****i 3
Troels Henriksen a****s@s****k 3
Harry Askham a****m@g****m 2
Qiao Zhang z****c@g****m 2
Martin Kiefel mk@n****e 2
Harry Askham h****m@g****m 2
Cynthia Shen c****3@g****m 2
and 12 more...
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 33
  • Total pull requests: 87
  • Average time to close issues: 9 months
  • Average time to close pull requests: 29 days
  • Total issue authors: 15
  • Total pull request authors: 13
  • Average comments per issue: 2.36
  • Average comments per pull request: 0.31
  • Merged pull requests: 69
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Top Authors
Issue Authors
  • axch (8)
  • dougalm (4)
  • duvenaud (4)
  • alexreinking (3)
  • apaszke (2)
  • darrenjw (2)
  • dan-zheng (2)
  • SalamanderXing (1)
  • martiningram (1)
  • general-rishkin (1)
  • jordanrule (1)
  • cartazio (1)
  • codereport (1)
  • Micduffy001 (1)
Pull Request Authors
  • axch (40)
  • dougalm (25)
  • duvenaud (7)
  • alexreinking (4)
  • ararslan (2)
  • emilyfertig (2)
  • dan-zheng (2)
  • MangoIV (2)
  • normanrink (2)
  • athas (1)
  • vilchy (1)
  • ComputerScientistDanny (1)
  • jagtesh (1)
Top Labels
Issue Labels
language / concrete syntax (3) tooling / editors (2) tooling / notebooks (1) bug (1) good first issue (1) discussion (1) language / compiler (1) backend (1) language / type system (1)
Pull Request Labels
tooling / notebooks (1)

Dependencies

dex.cabal hackage
  • hspec-discover * build
  • aeson *
  • ansi-terminal *
  • ansi-wl-pprint *
  • base *
  • binary *
  • blaze-html *
  • bytestring *
  • cmark *
  • containers *
  • cryptonite *
  • dex *
  • directory *
  • exceptions *
  • filepath *
  • hashable *
  • haskeline *
  • http-types *
  • llvm-hs *
  • llvm-hs-pure *
  • megaparsec *
  • mtl *
  • optparse-applicative *
  • parser-combinators *
  • prettyprinter *
  • process *
  • scientific *
  • store *
  • temporary *
  • text *
  • transformers *
  • unix *
  • unordered-containers *
  • utf8-string *
  • wai *
  • warp *
  • QuickCheck * test
  • base * test
  • dex * test
  • hspec * test
python/setup.py pypi
  • numpy *