Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of daphne-lib

daphne-lib - DAPHNE 0.3

Hereby we release the third development snapshot of DAPHNE containing many improvements and bug fixes that have been contributed by members of the DAPHNE consortium as well as external contributors (mentioned below). Find some of the more notable (bigger) changes below in the detailed release notes. Besides the artifact downloads on this page and the Docker images, we are releasing the python bindings daphne-lib on PyPI

Release Highlights

  • DaphneDSL/DaphneLib:
    • Additional ready-to-use data science algorithms: decision trees, random forests, PageRank
    • Various DaphneDSL language improvements (e.g., UDFs with multiple return values, more built-in functions, bounds-checks for left/right indexing, ...)
    • Support for complex control flow in DaphneLib
    • DaphneLib as a Python package
    • Efficient data exchange with pandas, TensorFlow, PyTorch
  • DAPHNE Compiler:
    • Initial MLIR-based codegen pipeline
    • Introduction of a kernel catalog to make the compiler aware of available pre-compiled kernels
  • DAPHNE Runtime:
    • CUDA 12.2.2
    • CUDA-support for more kernels
    • Support for synchronous gRPC and chunked data transfer in distributed runtime
    • AUTO option to the DAPHNE’s scheduling algorithms
  • Infrastructure and general:
    • Initial extensibility for custom kernels
    • More consistent and actionable error messages
    • Numerous little improvements and bug fixes

Contributors to this release

  • AlexRTer
  • Aristotelis Vontzalidis
  • Benjamin Steinwender
    • Constantin Pestka
    • DamianDinoiu
  • Daniel Wetzel
  • Garic
  • Henri Willems
  • Jonas Henrique Muller Korndorfer
  • Lachezar Nikolov
  • Lorenz Dirry
  • Marcus Paradies
  • Marius Birkenbach
  • Mark Dokter
  • Patrick Damme
  • Philipp Ortner
  • Quentin Guilloteau
  • Samin
  • StoeckOverflow
  • Stratos Psomadakis
  • Tom Schwarzburg
  • inikokali

Instructions to download and verify

``` bash

The signing keys

wget -qO- https://raw.githubusercontent.com/daphne-eu/daphne/main/KEYS.txt | gpg --import ```

``` bash

The plain DAPHNE release (ARMv8 aka 64 bit ARM)

wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-ARMV8-0.3-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-ARMV8-0.3-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-ARMV8-0.3-bin.tgz sha512sum -c daphne-ARMV8-0.3-bin.tgz.sha512sum gpg --verify daphne-ARMV8-0.3-bin.tgz.asc tar xf daphne-ARMV8-0.3-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-ARMV8-0.3-bin/lib:$LDLIBRARYPATH bash

The plain DAPHNE release (X86-64)

wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-X86-64-0.3-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-X86-64-0.3-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-X86-64-0.3-bin.tgz sha512sum -c daphne-X86-64-0.3-bin.tgz.sha512sum gpg --verify daphne-X86-64-0.3-bin.tgz.asc tar xf daphne-X86-64-0.3-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-X86-64-0.3-bin/lib:$LDLIBRARYPATH bash

The DAPHNE release with CUDA support (X86-64)

wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-cuda-X86-64-0.3-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-cuda-X86-64-0.3-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-cuda-X86-64-0.3-bin.tgz sha512sum -c daphne-cuda-X86-64-0.3-bin.tgz.sha512sum gpg --verify daphne-cuda-X86-64-0.3-bin.tgz.asc tar xf daphne-cuda-X86-64-0.3-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-cuda-X86-64-0.3-bin/lib:$LDLIBRARYPATH ```

What's Changed

  • Fix erroneous special treatment of default logger by @MarcusParadies in https://github.com/daphne-eu/daphne/pull/594
  • Cc cuda ops by @corepointer in https://github.com/daphne-eu/daphne/pull/596
  • Misc improvements by @corepointer in https://github.com/daphne-eu/daphne/pull/604
  • Fix passing str variable to readFrame/readMatrix by @MarcusParadies in https://github.com/daphne-eu/daphne/pull/598
  • [DAPHNE-#554] Use hwloc to query CPU topology by @psomas in https://github.com/daphne-eu/daphne/pull/563
  • [doc] add more docs regarding script arguments by @m-birke in https://github.com/daphne-eu/daphne/pull/620
  • [DAPHNE-#606] Some INT/FLOAT literal suffixes from C++ in DaphneDSL by @corepointer in https://github.com/daphne-eu/daphne/pull/606
  • 522 DaphneLib complex control flow and lazy evaluated functions by @lachezar-n in https://github.com/daphne-eu/daphne/pull/574
  • 615-ConstantfoldingEwlogOp by @inikokali in https://github.com/daphne-eu/daphne/pull/630
  • [MINOR] Enable shape inference for frame type by @DamianDinoiu in https://github.com/daphne-eu/daphne/pull/626
  • [MINOR] Parsing IR fix. by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/634
  • MLIR-based code generation pipeline by @philipportner in https://github.com/daphne-eu/daphne/pull/633
  • SQLParser: Introduce DISTINCT clause by @tomschw in https://github.com/daphne-eu/daphne/pull/564
  • SQLParser: Support for SELECT , SELECT f., GROUP BY , GROUP BY f. by @tomschw in https://github.com/daphne-eu/daphne/pull/588
  • 613-Support missing aggregation functions in aggregation kernels by @inikokali in https://github.com/daphne-eu/daphne/pull/638
  • Integration of mpi_testing into test.sh by @inikokali in https://github.com/daphne-eu/daphne/pull/639
  • Close daphne-eu/daphne#614 Support missing elementwise unary functions in elementwise unary kernels by @AlexRTer in https://github.com/daphne-eu/daphne/pull/644
  • [doc] fix typo in links within Readme by @AlexRTer in https://github.com/daphne-eu/daphne/pull/645
  • Automated task grain size calculation (related to issue 567). by @jhmkorndorfer in https://github.com/daphne-eu/daphne/pull/647
  • #650 fix get topology by @GuilloteauQ in https://github.com/daphne-eu/daphne/pull/651
  • [DAPHNE-#572] Bounds checks for left and right indexing by @AlexRTer in https://github.com/daphne-eu/daphne/pull/648
  • Distributed worker: Insert IR arguments to body. by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/623
  • [DAPHNE-#669] properly handle views in WriteCsv kernel by @AlexRTer in https://github.com/daphne-eu/daphne/pull/678
  • Feature/daphnelib python pkg by @m-birke in https://github.com/daphne-eu/daphne/pull/649
  • Pandas, Pytorch & Tensorflow Shared Memory Support (Issue #499) by @danielwetzel in https://github.com/daphne-eu/daphne/pull/585
  • [DAPHNE-#687] OneHot bound check by @AlexRTer in https://github.com/daphne-eu/daphne/pull/695
  • Code Generation for Matrix Multiplication by @resting-dove in https://github.com/daphne-eu/daphne/pull/653
  • Improve error handling during compilation + runtime by @philipportner in https://github.com/daphne-eu/daphne/pull/706
  • Basic extension catalog for kernels, plus demo on early extensibility. by @pdamme in https://github.com/daphne-eu/daphne/pull/456
  • Generic Matrix kernel specializations by @AlexRTer in https://github.com/daphne-eu/daphne/pull/722
  • [BUGFIX] Build target dependencies. by @pdamme in https://github.com/daphne-eu/daphne/pull/728
  • [DAPHNE-#531] Simplified GettingStarted.md. by @pdamme in https://github.com/daphne-eu/daphne/pull/725
  • [DAPHNE-#659] Scientific notation for float literals and digit separators by @AlexRTer in https://github.com/daphne-eu/daphne/pull/677
  • Tensor Support - 1: Data structure by @CPestka in https://github.com/daphne-eu/daphne/pull/694
  • Non-deterministic test fail during Github CI action by @philipportner in https://github.com/daphne-eu/daphne/pull/733
  • Enable MathJax for documentation by @AlexRTer in https://github.com/daphne-eu/daphne/pull/726
  • [DAPHNE-#747] Use test.sh in CI again. by @pdamme in https://github.com/daphne-eu/daphne/pull/748
  • [DAPHNE-#523] DaphneDSL flexible matrix/frame literals by @AlexRTer in https://github.com/daphne-eu/daphne/pull/675
  • daphnelib: improve pyproject.toml and readme by @m-birke in https://github.com/daphne-eu/daphne/pull/743
  • Update docs.yml by @auge in https://github.com/daphne-eu/daphne/pull/750
  • Update main.yml by @auge in https://github.com/daphne-eu/daphne/pull/751
  • Add missing (local) kernel tests by @AlexRTer in https://github.com/daphne-eu/daphne/pull/730
  • [DAPHNE-#658] stop() built-in/op/kernel by @Garic152 in https://github.com/daphne-eu/daphne/pull/753
  • [DAPHNE-#666] optional inc for seq function by @saminbassiri in https://github.com/daphne-eu/daphne/pull/756
  • mac support ongoing by @auge in https://github.com/daphne-eu/daphne/pull/764
  • 752 one hot and bin functions in daphne lib by @saminbassiri in https://github.com/daphne-eu/daphne/pull/760
  • [DAPHNE-#680] bug fix for signed integer casts by @AlexRTer in https://github.com/daphne-eu/daphne/pull/744
  • [security] Remove polyfill.io dependency in docs by @AlexRTer in https://github.com/daphne-eu/daphne/pull/769
  • #720 replace asserts in codebase with exceptions by @AlexRTer in https://github.com/daphne-eu/daphne/pull/732
  • Add AggAll kernels for missing value types by @resting-dove in https://github.com/daphne-eu/daphne/pull/674
  • Address Wrong results using Integer Matmul Kernels after Matrix resizing by @resting-dove in https://github.com/daphne-eu/daphne/pull/701
  • Refactor getLine() to avoid potential memory leak by @psomas in https://github.com/daphne-eu/daphne/pull/770
  • [DAPHNE-#560] Parsing unary minus operator by @corepointer in https://github.com/daphne-eu/daphne/pull/607
  • [DAPHNE-#595] Fix crash when calling print(f()) with empty function by @corepointer in https://github.com/daphne-eu/daphne/pull/605
  • [DAPHNE-#768] isNan: elementwise check for NaN elements in a matrix - Draft PR by @StoeckOverflow in https://github.com/daphne-eu/daphne/pull/779
  • Update software requirements in doc by @AlexRTer in https://github.com/daphne-eu/daphne/pull/786
  • 657 DaphneLib implement indexing by @saminbassiri in https://github.com/daphne-eu/daphne/pull/771
  • Implementation of new SparsityOp by @Garic152 in https://github.com/daphne-eu/daphne/pull/790
  • Improve performance of the transposition of CSR matrices by @GuilloteauQ in https://github.com/daphne-eu/daphne/pull/798
  • 788 closing mem leaks by @corepointer in https://github.com/daphne-eu/daphne/pull/791
  • [DAPHNE-#774] Simplification rewrites for elementwise unary minus (ad… by @ldirry in https://github.com/daphne-eu/daphne/pull/796
  • [DAPHNE-#775] Support for unary minus in DaphneLib by @ldirry in https://github.com/daphne-eu/daphne/pull/792
  • [DAPHNE-#755] Initial support for lists in DaphneDSL. by @pdamme in https://github.com/daphne-eu/daphne/pull/806
  • Implementation of CSR/CSR Matrix Multiplication (@) by @GuilloteauQ in https://github.com/daphne-eu/daphne/pull/799
  • [DAPHNE-#773] Undetected invalid script args by @ldirry in https://github.com/daphne-eu/daphne/pull/800

New Contributors

  • @MarcusParadies made their first contribution in https://github.com/daphne-eu/daphne/pull/594
  • @lachezar-n made their first contribution in https://github.com/daphne-eu/daphne/pull/574
  • @inikokali made their first contribution in https://github.com/daphne-eu/daphne/pull/630
  • @AlexRTer made their first contribution in https://github.com/daphne-eu/daphne/pull/644
  • @jhmkorndorfer made their first contribution in https://github.com/daphne-eu/daphne/pull/647
  • @GuilloteauQ made their first contribution in https://github.com/daphne-eu/daphne/pull/651
  • @danielwetzel made their first contribution in https://github.com/daphne-eu/daphne/pull/585
  • @resting-dove made their first contribution in https://github.com/daphne-eu/daphne/pull/653
  • @CPestka made their first contribution in https://github.com/daphne-eu/daphne/pull/694
  • @Garic152 made their first contribution in https://github.com/daphne-eu/daphne/pull/753
  • @saminbassiri made their first contribution in https://github.com/daphne-eu/daphne/pull/756
  • @StoeckOverflow made their first contribution in https://github.com/daphne-eu/daphne/pull/779
  • @ldirry made their first contribution in https://github.com/daphne-eu/daphne/pull/796

Full Changelog: https://github.com/daphne-eu/daphne/compare/0.2...0.3

- C++
Published by corepointer over 1 year ago

daphne-lib - DAPHNE 0.3-rc1

Second release candidate for v0.3 containing a number of bug fixes and improvements. This time making use of the "generate release notes" feature. Please test and report (in the form of GitHub issues)

What's Changed

  • 657 DaphneLib implement indexing by @saminbassiri in https://github.com/daphne-eu/daphne/pull/771
  • Implementation of new SparsityOp by @Garic152 in https://github.com/daphne-eu/daphne/pull/790
  • Improve performance of the transposition of CSR matrices by @GuilloteauQ in https://github.com/daphne-eu/daphne/pull/798
  • 788 closing mem leaks by @corepointer in https://github.com/daphne-eu/daphne/pull/791
  • [DAPHNE-#774] Simplification rewrites for elementwise unary minus (ad… by @ldirry in https://github.com/daphne-eu/daphne/pull/796
  • [DAPHNE-#775] Support for unary minus in DaphneLib by @ldirry in https://github.com/daphne-eu/daphne/pull/792
  • [DAPHNE-#755] Initial support for lists in DaphneDSL. by @pdamme in https://github.com/daphne-eu/daphne/pull/806
  • Implementation of CSR/CSR Matrix Multiplication (@) by @GuilloteauQ in https://github.com/daphne-eu/daphne/pull/799
  • [DAPHNE-#773] Undetected invalid script args by @ldirry in https://github.com/daphne-eu/daphne/pull/800

New Contributors

  • @ldirry made their first contribution in https://github.com/daphne-eu/daphne/pull/796

Full Changelog: https://github.com/daphne-eu/daphne/compare/0.3-rc0...0.3-rc1

- C++
Published by corepointer over 1 year ago

daphne-lib - DAPHNE 0.3-rc0

The first release candidate of version 0.3. Please test and report.

- C++
Published by corepointer over 1 year ago

daphne-lib - DAPHNE v0.2

Hereby we release the second development snapshot of DAPHNE containing many improvements and bug fixes that have been contributed by members of the DAPHNE consortium as well as external contributors (mentioned below). Find some of the more notable (bigger) changes below in the detailed release notes. Besides the artifact downloads on this page, new Docker images are available on dockerhub.

Release Highlights

  • DaphneDSL: various little additions and improvements
    • second-order function map()
    • elementwise conditional operator
  • system internals
    • initial OpenMPI backend for distributed runtime
    • some more FPGA kernels
    • improvements to the vectorized engine
    • logging facility based on spdlog
    • initial profiling features
  • build and deployment: various improvements
    • containers for simple deployment
    • refactoring of the build scripts
    • continuous integration
  • external dependencies
    • libeigen is used for calculating Eigen values/vectors and for integer matrix multiply
    • libpapi for profiling
    • OpenMPI for distributed operation
    • hwloc for detecting processor details
    • spdlog for loggin
  • miscellaneous changes
    • based on a new snapshot of LLVM/MLIR
    • CUDA version 12.1.1
    • support for 64 bit ARM (CPU only)

Contributors

This is an alphabetically sorted list (based on git history) of people who contributed to this release: * Ahmed Eleliemy * Aleš Zamuda * Aristotelis Vontzalidis * Benjamin Steinwender * Damian Dinoiu * Dzevad Coralic * Eric Mier * Kostas Bitsakos * Marius Birkenbach * Mark Dokter * Patrick Damme * Piotr Ratuszniak * Simeon * Stratos Psomadakis * Tom Schwarzburg

What's Changed

  • [DAHPNE-#198] Second order function map by @sigmaeon in https://github.com/daphne-eu/daphne/pull/407
  • Fpga operators by @ratusz in https://github.com/daphne-eu/daphne/pull/451
  • 459: enable CI by @auge in https://github.com/daphne-eu/daphne/pull/460
  • Upgraded MLIR. by @pdamme in https://github.com/daphne-eu/daphne/pull/468
  • Cached GitHub docker action by @corepointer in https://github.com/daphne-eu/daphne/pull/470
  • Build script improvements by @corepointer in https://github.com/daphne-eu/daphne/pull/471
  • Docker improvements by @corepointer in https://github.com/daphne-eu/daphne/pull/472
  • Draft: [DAPHNE-#132] Frame column labels after group-join by @DamianDinoiu in https://github.com/daphne-eu/daphne/pull/478
  • Duplicate inputs in pipeline input array. by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/476
  • Stripping et al by @corepointer in https://github.com/daphne-eu/daphne/pull/490
  • Docker cuda by @corepointer in https://github.com/daphne-eu/daphne/pull/491
  • [DAPHNE-#492] Creating Subframe of Frame with zero rows by @tomschw in https://github.com/daphne-eu/daphne/pull/496
  • Metadata sharing gpu fixes by @corepointer in https://github.com/daphne-eu/daphne/pull/503
  • Fix: Force set the cached variable ANTLR4JARLOCATION, if old path not exists by @EricMier in https://github.com/daphne-eu/daphne/pull/506
  • Added load partioning for distributed runtime. by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/489
  • DaphneLib Frames implementation by @daphne-eu in https://github.com/daphne-eu/daphne/pull/318
  • Eigen op 391 by @KostasBitsakos in https://github.com/daphne-eu/daphne/pull/463
  • CUDA fill by @daphne-eu in https://github.com/daphne-eu/daphne/pull/313
  • Create MPI-Usage.md by @AhmedEleliemy in https://github.com/daphne-eu/daphne/pull/532
  • [DAPHNE-#427] Initial logging facility using spdlog by @corepointer in https://github.com/daphne-eu/daphne/pull/535
  • [DAPHNE-#530] Docker Improvements by @corepointer in https://github.com/daphne-eu/daphne/pull/534
  • Add profiling support by @psomas in https://github.com/daphne-eu/daphne/pull/479
  • Dev-issue-507-dependency-path-changes by @EricMier in https://github.com/daphne-eu/daphne/pull/508
  • Eigen op 391 by @KostasBitsakos in https://github.com/daphne-eu/daphne/pull/539
  • [DAPHNE-#493] SQLParser: Run Aggregation without grouping by @tomschw in https://github.com/daphne-eu/daphne/pull/509
  • [DAPHNE-#540] Support for the ARMV8 CPU architecture by @corepointer in https://github.com/daphne-eu/daphne/pull/540
  • [DAPHNE-#399] Aggregation kernels can return different value types by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/402
  • Initial implementation of DaphneSerializer by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/467
  • [DAPHNE-#494] SQLParser: Group by without aggregation clause not working by @tomschw in https://github.com/daphne-eu/daphne/pull/543
  • Logger fixes by @corepointer in https://github.com/daphne-eu/daphne/pull/547
  • Integer matrix multiplication #352 by @KostasBitsakos in https://github.com/daphne-eu/daphne/pull/384
  • Feature/docs mvp by @m-birke in https://github.com/daphne-eu/daphne/pull/550
  • [DOC] Improvements to quickstart guide. by @pdamme in https://github.com/daphne-eu/daphne/pull/566
  • Pass errors are not only logged, but also printed again. by @pdamme in https://github.com/daphne-eu/daphne/pull/571
  • [doc] fix md flaws and add newely added docs to html page by @m-birke in https://github.com/daphne-eu/daphne/pull/570
  • [DAPHNE-#575] Error logger, meaningful default log level, cleanup by @corepointer in https://github.com/daphne-eu/daphne/pull/575

New Contributors

  • @sigmaeon made their first contribution in https://github.com/daphne-eu/daphne/pull/407
  • @auge made their first contribution in https://github.com/daphne-eu/daphne/pull/460
  • @DamianDinoiu made their first contribution in https://github.com/daphne-eu/daphne/pull/478
  • @m-birke made their first contribution in https://github.com/daphne-eu/daphne/pull/550

Full Changelog: https://github.com/daphne-eu/daphne/compare/0.1...0.2

Instructions to download and verify

``` bash

The signing keys

wget -qO- https://raw.githubusercontent.com/daphne-eu/daphne/main/KEYS.txt | gpg --import

The plain DAPHNE X86-64 release

wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-X86-64-v0.2-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-X86-64-v0.2-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-X86-64-v0.2-bin.tgz sha512sum -c daphne-X86-64-v0.2-bin.tgz.sha512sum gpg --verify daphne-X86-64-v0.2-bin.tgz.asc tar xf daphne-X86-64-v0.2-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-X86-64-v0.2-bin/lib:$LDLIBRARYPATH

The DAPHNE release with CUDA support

wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-cuda-X86-64-v0.2-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-cuda-X86-64-v0.2-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-cuda-X86-64-v0.2-bin.tgz sha512sum -c daphne-cuda-X86-64-v0.2-bin.tgz.sha512sum gpg --verify daphne-cuda-X86-64-v0.2-bin.tgz.asc tar xf daphne-cuda-X86-64-v0.2-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-cuda-X86-64-v0.2-bin/lib:$LDLIBRARYPATH

The DAPHNE release with ARM support

wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-ARMV8-v0.2-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-ARMV8-v0.2-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.2/daphne-ARMV8-v0.2-bin.tgz sha512sum -c daphne-ARMV8-v0.2-bin.tgz.sha512sum gpg --verify daphne-ARMV8-v0.2-bin.tgz.asc tar xf daphne-ARMV8-v0.2-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-ARMV8-v0.2-bin/lib:$LDLIBRARYPATH ```

- C++
Published by corepointer over 2 years ago

daphne-lib - Release 0.1

Initial Release of DAPHNE

This first release contains basic initial implementations of most components of the DAPHNE system infrastructure, from a domain-specific language (DaphneDSL) as the main user frontend so far (a Python API is under development), over an MLIR-based intermediate representation (DaphneIR) and compilation chain to runtime components for the distributed execution using a stand-alone backend based on gRPC and the local, optionally vectorized, execution on CPU, GPU, and FPGA, for a certain subset of operations and data/value types.

Release Notes:

  • The focus of this release is on providing an early preview of the DAPHNE system prototype for interested researchers/developers/users.
  • We are aware of several bugs and issues. Note also that several features of the system are still experimental.
  • Likewise, there is still a lot of room for performance improvements in various components.
  • The binary release contains precompiled DAPHNE libraries, executables, scripts, and documentation.
  • Binaries were generated on Ubuntu 20.04 LTS on an Intel Xeon machine compiled for x86_64.
  • GPU operations were compiled against CUDA 11.7.1.
  • As this is the first release, the auto-generated full changelog contains all commits to date.
  • Experimental features (FPGA, Arrow) which are not available in the binary release can be compiled from source. Follow the instructions in the documentation section.
  • To avoid problems with library dependencies, two binary artifacts are provided for this release. One compiled with CUDA support and one without.

Contributors:

This is an alphabetically sorted list (based on git history) of people who contributed to this release:

  • Ahmed Eleliemy
  • Alexander Hiebl
  • Aleš Zamuda
  • Aristotelis Vontzalidis
  • Artem Kroviakov
  • Constantin Pestka
  • Dominic Schablas
  • Dževad Ćoralić
  • Eric Mier
  • Gabrielle Poerwawinata
  • Izajasz Wrosz
  • Jonathan Giger
  • Kevin Innerebner
  • Konstantinos Bitsakos
  • Lennart Schmidt
  • Mark Dokter
  • Matej Moravec
  • Matthias Boehm
  • Morten Tychsen Clausen
  • Niclas Hedam
  • Patrick Damme
  • Philipp Ortner
  • Philippe Bonnet
  • Piotr Ratuszniak
  • Stratos Psomadakis

Instructions to download and verify

``` bash

The signing keys

wget -qO- https://raw.githubusercontent.com/daphne-eu/daphne/main/KEYS.txt | gpg --import

The plain DAPHNE release

wget https://github.com/daphne-eu/daphne/releases/download/0.1/daphne-0.1-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.1/daphne-0.1-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.1/daphne-0.1-bin.tgz sha512sum -c daphne-0.1-bin.tgz.sha512sum gpg --verify daphne-0.1-bin.tgz.asc tar xf daphne-0.1-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne-0.1-bin/lib:$LDLIBRARYPATH

The DAPHNE release with CUDA support

wget https://github.com/daphne-eu/daphne/releases/download/0.1/daphne--cuda-0.1-bin.tgz.sha512sum wget https://github.com/daphne-eu/daphne/releases/download/0.1/daphne--cuda-0.1-bin.tgz.asc wget https://github.com/daphne-eu/daphne/releases/download/0.1/daphne--cuda-0.1-bin.tgz sha512sum -c daphne--cuda-0.1-bin.tgz.sha512sum gpg --verify daphne--cuda-0.1-bin.tgz.asc tar xf daphne--cuda-0.1-bin.tgz

add lib directory to environment

export LDLIBRARYPATH=$PWD/daphne--cuda-0.1-bin/lib:$LDLIBRARYPATH ```

What's Changed

  • [DAPHNE-#216] ANTLR not found while building by @daphne-eu in https://github.com/daphne-eu/daphne/pull/332
  • [DAPHNE-#128] Kernel for CastOp from DenseMatrix of one value type to DenseMatrix of another value type. by @akroviakov in https://github.com/daphne-eu/daphne/pull/337
  • Added reading the prototype configuration from JSON file by @matejmoravec in https://github.com/daphne-eu/daphne/pull/336
  • [DAPHNE-340] Fix FISS chunk size calculation by @jonnygiger in https://github.com/daphne-eu/daphne/pull/341
  • 236 copying deploy files to merge by @aleszamuda in https://github.com/daphne-eu/daphne/pull/335
  • New IO kernels: Matrix Market, Parquet and DAPHNE Binary by @daphne-eu in https://github.com/daphne-eu/daphne/pull/331
  • [DAPHNE-#234] extractCol-kernel for Frame with positions in DenseMatr… by @Hiebl in https://github.com/daphne-eu/daphne/pull/338
  • [DAPHNE-#217]: rowBind-kernel for CSRMatrix by @akroviakov in https://github.com/daphne-eu/daphne/pull/344
  • [DAPHNE-#218]: colBind-kernel for CSRMatrix by @akroviakov in https://github.com/daphne-eu/daphne/pull/347
  • [DAPHNE-#224]: reshape-kernel for non-zero-copy cases by @akroviakov in https://github.com/daphne-eu/daphne/pull/348
  • [DAPHNE-#215]: Ternary/conditional operator ?: in DaphneDSL by @akroviakov in https://github.com/daphne-eu/daphne/pull/349
  • [DAPHNE-#31] Kernel for diagMatrix-operation by @psomas in https://github.com/daphne-eu/daphne/pull/362
  • [DAPHNE-#350]: DaphneDSL: rbind checks number of rows instead the number of columns by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/360
  • [DAPHNE-#356]: type casting of DenseMatrix by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/361
  • Support for mean operation for all-aggregation and row-aggregation. by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/398
  • [DAPHNE-#358]: DaphneDSL: Matrix literals by @akroviakov in https://github.com/daphne-eu/daphne/pull/387
  • [DAPHNE-#390]: Kernel for ctable operation by @akroviakov in https://github.com/daphne-eu/daphne/pull/392
  • Draft: [DAPHNE-#242] Kernel for GroupOp on Frame by @Hiebl in https://github.com/daphne-eu/daphne/pull/363
  • [DAPHNE-#403]: CLI args for explaining/printing the IR at various points by @philipportner in https://github.com/daphne-eu/daphne/pull/404
  • CSRMatrix to Dense Matrix CastOpt and vise versa by @KostasBitsakos in https://github.com/daphne-eu/daphne/pull/366
  • Enable Arrow by flag + arrow docs by @niclashedam in https://github.com/daphne-eu/daphne/pull/345
  • [DAPHNE-#230]: String as value type of DenseMatrix by @akroviakov in https://github.com/daphne-eu/daphne/pull/381
  • [DAPHNE-#388] order-kernel for DenseMatrix by @Hiebl in https://github.com/daphne-eu/daphne/pull/400
  • [DAPHNE-#389] order-kernel: return indexes instead of ordered data object by @Hiebl in https://github.com/daphne-eu/daphne/pull/411
  • [DAPHNE-196] Multiple task queues and work stealing by @jonnygiger in https://github.com/daphne-eu/daphne/pull/342
  • [DAPHNE-191] Refactor CUDA buffer mgmt (aka introducing Object Meta D… by @corepointer in https://github.com/daphne-eu/daphne/pull/334
  • [DAPHNE-#420]: ANTLR Any Conversion. by @DerSchmidt in https://github.com/daphne-eu/daphne/pull/422
  • [DAPHNE-#200] Transposition-aware matrix multiplication by @Hiebl in https://github.com/daphne-eu/daphne/pull/406
  • Distributed pipelines by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/428
  • [DAPHNE-#419] Guidelines on handling pull requests by @pdamme in https://github.com/daphne-eu/daphne/pull/430
  • Fpga operators by @ratusz in https://github.com/daphne-eu/daphne/pull/416
  • [DAPHNE-#214]: Import mechanism in DaphneDSL by @akroviakov in https://github.com/daphne-eu/daphne/pull/410
  • [DAPHNE-#55]: Casts in DaphneDSL by @akroviakov in https://github.com/daphne-eu/daphne/pull/408
  • [DAPHNE-#417] Incorrect GROUP BY result when all keys are the same by @Hiebl in https://github.com/daphne-eu/daphne/pull/426
  • [DAPHNE-#385] Connected JSON-based file meta data parser to file IO by @matejmoravec in https://github.com/daphne-eu/daphne/pull/394
  • Documentation of scheduling options and extensions by @AhmedEleliemy in https://github.com/daphne-eu/daphne/pull/435
  • Integration of ThetaJoin and OrderBy kernels into SQL Parser + Lowering by @EricMier in https://github.com/daphne-eu/daphne/pull/440
  • [DAPHNE-#118] Initial write-up of the DaphneDSL documentation. by @pdamme in https://github.com/daphne-eu/daphne/pull/443
  • Distributed runtime documentation. by @aristotelis96 in https://github.com/daphne-eu/daphne/pull/431
  • [DAPHNE-#437] Deployment documentation (containerized packaging, virtualized installation, managed deployment, and distributed execution) by @aleszamuda in https://github.com/daphne-eu/daphne/pull/441
  • SQL tutorial by @DerSchmidt in https://github.com/daphne-eu/daphne/pull/442
  • Release scripts by @corepointer in https://github.com/daphne-eu/daphne/pull/444
  • [DAPHNE-#452] Release script feature parameter by @corepointer in https://github.com/daphne-eu/daphne/pull/452

New Contributors

  • @daphne-eu made their first contribution in https://github.com/daphne-eu/daphne/pull/332
  • @akroviakov made their first contribution in https://github.com/daphne-eu/daphne/pull/337
  • @matejmoravec made their first contribution in https://github.com/daphne-eu/daphne/pull/336
  • @jonnygiger made their first contribution in https://github.com/daphne-eu/daphne/pull/341
  • @aleszamuda made their first contribution in https://github.com/daphne-eu/daphne/pull/335
  • @Hiebl made their first contribution in https://github.com/daphne-eu/daphne/pull/338
  • @psomas made their first contribution in https://github.com/daphne-eu/daphne/pull/362
  • @aristotelis96 made their first contribution in https://github.com/daphne-eu/daphne/pull/360
  • @philipportner made their first contribution in https://github.com/daphne-eu/daphne/pull/404
  • @KostasBitsakos made their first contribution in https://github.com/daphne-eu/daphne/pull/366
  • @niclashedam made their first contribution in https://github.com/daphne-eu/daphne/pull/345
  • @corepointer made their first contribution in https://github.com/daphne-eu/daphne/pull/334
  • @DerSchmidt made their first contribution in https://github.com/daphne-eu/daphne/pull/422
  • @pdamme made their first contribution in https://github.com/daphne-eu/daphne/pull/430
  • @ratusz made their first contribution in https://github.com/daphne-eu/daphne/pull/416
  • @AhmedEleliemy made their first contribution in https://github.com/daphne-eu/daphne/pull/435

Full Changelog: https://github.com/daphne-eu/daphne/commits/0.1

- C++
Published by corepointer over 3 years ago