dex-lang
Research language for array processing in the Haskell/ML family
Science Score: 54.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.0%) to scientific vocabulary
Keywords from Contributors
Repository
Research language for array processing in the Haskell/ML family
Basic Info
Statistics
- Stars: 1,617
- Watchers: 56
- Forks: 110
- Open Issues: 141
- Releases: 0
Metadata Files
README.md
Dex 
Dex (named for "index") is a research language for typed, functional array processing. The goal of the project is to explore:
- Type systems for array programming
- Mathematical program transformations like differentiation and integration
- User-directed compilation to parallel hardware
- Interactive and incremental numerical programming and visualization
To learn more, check out our paper, our tutorial or these example programs:
- Dex prelude
- Mandelbrot set
- Ray tracer
- Estimating pi
- Hamiltonian Monte Carlo
- ODE integrator
- Sierpinski triangle
- Basis function regression
- Brownian bridge
- Dynamic programming (Levenshtein distance)
- Molecular dynamics simulation
Or for a more comprehensive look, there's
- The InDex of all documents, libraries, and examples included in this repository.
🚨 Dex is an experimental research project at an early stage of development. Expect monstrous bugs and razor-sharp edges!
🤝 Contributions welcome! See our issue tracker for good first issues, or browse by thematic labels.
Dependencies
- Install stack
- Install LLVM 12
- Ubuntu/Debian:
apt-get install llvm-12-dev - macOS:
brew install llvm@12- Make sure
llvm@12is on yourPATHbefore building. Example:export PATH="$(brew --prefix llvm@12)/bin:$PATH"
- Make sure
- Ubuntu/Debian:
- Install clang 12 (may be installed together with llvm)
- Ubuntu/Debian:
apt-get install clang-12 - macOS: installs with llvm
- Ubuntu/Debian:
- Install libpng (often included by default in *nix platforms)
- Ubuntu/Debian:
apt-get install libpng-dev - macOS:
brew install libpng
- Ubuntu/Debian:
Installing
To build and install a release version of Dex run make install.
The default installation directory is $HOME/.local/bin, so make sure to add
that directory to $PATH after installing Dex. To install Dex somewhere else,
set the PREFIX environment variable before running make install. For
example, PREFIX=$HOME make install installs dex in $HOME/bin.
Building
- Build Dex in development (unoptimized) mode:
make - Run tests in development mode:
make tests
It is convenient to set up a dex alias (e.g. in .bashrc) for running Dex in
development mode:
```console
Linux:
alias dex="stack exec dex -- --lib-path lib"
macOS:
alias dex="stack exec --stack-yaml=stack-macos.yaml dex -- --lib-path lib" ```
You might also want to consider other development build targets:
* build-opt for local optimized builds,
* build-dbg for local debug builds,
* build-prof for local optimized builds with profiling enabled.
Those non-standard targets require different aliases. Please consult the documentation
at the top of the makefile
for detailed instructions.
Haskell Language Server
In order to use HLS with the Haskell code in this project:
- Install ghcup
- Run
ghcup install hls - Create a file in the root
dexdirectory calledhie.yamlwith the following contents:
yaml
cradle:
stack:
stackYaml: "./stack-macos.yaml" # Or stack.yaml if not on MacOS
Unfortunately one cannot dynamically select the stack.yaml file to use based
on the environment, and so one has to create an appropriate hie.yaml file
manually. This will be ignored by git.
This should work out of the box with Emacs' lsp-haskell package.
Building with Nix
Nix is a functional package manager and build system.
To build with vanilla Nix:
bash
$ nix-build
To build with flakes-enabled Nix:
bash
$ nix build .#dex
The resulting dex binary should be in result/bin/dex.
For development purposes, you can use a Nix environment with
bash
$ nix-shell
$ nix develop # With flakes
and use make to use Stack to build Dex.
Running
- Traditional REPL:
dex repl - Execute script:
dex script examples/pi.dx - Live-updated notebook display
dex web examples/pi.dx(html) ordex watch examples/pi.dx(terminal).
License
BSD-3
This is an early-stage research project, not an official Google product.
Owner
- Name: Google Research
- Login: google-research
- Kind: organization
- Location: Earth
- Website: https://research.google
- Repositories: 226
- Profile: https://github.com/google-research
Citation (CITATION.cff)
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- given-names: "Dougal"
family-names: "Maclaurin"
- given-names: "Adam"
family-names: "Paszke"
title: "Dex: typed and functional array processing"
version: 0.0.0
date-released: 2018-09-29
url: "https://github.com/google-research/dex-lang"
preferred-citation:
type: article
authors:
- given-names: "Adam"
family-names: "Paszke"
- given-names: "Daniel D."
family-names: "Johnson"
- given-names: "David"
family-names: "Duvenaud"
- given-names: "Dimitrios"
family-names: "Vytiniotis"
- given-names: "Alexey"
family-names: "Radul"
- given-names: "Matthew J."
family-names: "Johnson"
- given-names: "Jonathan"
family-names: "Ragan-Kelley"
- given-names: "Dougal"
family-names: "Maclaurin"
doi: "10.1145/3473593"
title: "Getting to the Point: Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming"
volume: 5
number: "ICFP"
year: 2021
month: 8
issue-date: "August 2021"
journal: "Proceedings of the ACM on Programming Languages"
pages: 29
abstract: >
We present a novel programming language design that attempts to combine the clarity
and safety of high-level functional languages with the efficiency and parallelism
of low-level numerical languages. We treat arrays as eagerly-memoized functions on
typed index sets, allowing abstract function manipulations, such as currying, to work
on arrays. In contrast to composing primitive bulk-array operations, we argue for
an explicit nested indexing style that mirrors application of functions to arguments.
We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized
in-place updates. Specifically, an associative accumulation effect allows reverse-mode
automatic differentiation of in-place updates in a way that preserves parallelism.
Empirically, we benchmark against the Futhark array programming language, and demonstrate
that aggressive inlining and type-driven compilation allows array programs to be written
in an expressive, "pointful" style with little performance penalty.
keywords:
- "array programming"
- "automatic differentiation"
- "parallel computing"
url: "https://doi.org/10.1145/3473593"
publisher:
name: "Association for Computing Machinery"
city: "New York"
region: "NY"
country: "USA"
GitHub Events
Total
- Issues event: 1
- Watch event: 51
- Push event: 14
- Fork event: 7
Last Year
- Issues event: 1
- Watch event: 51
- Push event: 14
- Fork event: 7
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| Dougal Maclaurin | d****m@g****m | 1,748 |
| Adam Paszke | a****e@g****m | 469 |
| Alexey Radul | a****h@g****m | 390 |
| David Duvenaud | d****d@g****m | 83 |
| Daniel Johnson | d****n@g****m | 72 |
| Lyndon White | l****e@i****k | 59 |
| Dan Zheng | d****g@g****m | 43 |
| srush | s****h@g****m | 29 |
| Norman Rink | n****k@g****m | 22 |
| Matthew Johnson | m****j@g****m | 18 |
| chenzizhao | z****o@g****m | 12 |
| Yonghao Jin | y****1@g****m | 11 |
| joaogui1 | j****o@g****m | 11 |
| Emily Fertig | e****f@g****m | 10 |
| Darren Wilkinson | d****n@b****m | 10 |
| Alex Reinking | r****g@g****m | 8 |
| Niklas Schmitz | n****z@g****m | 7 |
| DHHZ | d****g@g****m | 6 |
| Jonathan Merritt | j****t@g****m | 4 |
| Peter Becich | p****h@g****m | 4 |
| Sharad Vikram | s****m@g****m | 4 |
| Torsten Scholak | t****k@g****m | 4 |
| CJ Carey | p****e@g****m | 4 |
| Benjamin Chetioui | 3****i | 3 |
| Troels Henriksen | a****s@s****k | 3 |
| Harry Askham | a****m@g****m | 2 |
| Qiao Zhang | z****c@g****m | 2 |
| Martin Kiefel | mk@n****e | 2 |
| Harry Askham | h****m@g****m | 2 |
| Cynthia Shen | c****3@g****m | 2 |
| and 12 more... | ||
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 11 months ago
All Time
- Total issues: 33
- Total pull requests: 87
- Average time to close issues: 9 months
- Average time to close pull requests: 29 days
- Total issue authors: 15
- Total pull request authors: 13
- Average comments per issue: 2.36
- Average comments per pull request: 0.31
- Merged pull requests: 69
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 1
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 1
- Pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- axch (8)
- dougalm (4)
- duvenaud (4)
- alexreinking (3)
- apaszke (2)
- darrenjw (2)
- dan-zheng (2)
- SalamanderXing (1)
- martiningram (1)
- general-rishkin (1)
- jordanrule (1)
- cartazio (1)
- codereport (1)
- Micduffy001 (1)
Pull Request Authors
- axch (40)
- dougalm (25)
- duvenaud (7)
- alexreinking (4)
- ararslan (2)
- emilyfertig (2)
- dan-zheng (2)
- MangoIV (2)
- normanrink (2)
- athas (1)
- vilchy (1)
- ComputerScientistDanny (1)
- jagtesh (1)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- hspec-discover * build
- aeson *
- ansi-terminal *
- ansi-wl-pprint *
- base *
- binary *
- blaze-html *
- bytestring *
- cmark *
- containers *
- cryptonite *
- dex *
- directory *
- exceptions *
- filepath *
- hashable *
- haskeline *
- http-types *
- llvm-hs *
- llvm-hs-pure *
- megaparsec *
- mtl *
- optparse-applicative *
- parser-combinators *
- prettyprinter *
- process *
- scientific *
- store *
- temporary *
- text *
- transformers *
- unix *
- unordered-containers *
- utf8-string *
- wai *
- warp *
- QuickCheck * test
- base * test
- dex * test
- hspec * test
- numpy *