Hereby we release the third development snapshot of DAPHNE containing many improvements and bug fixes that have been contributed by members of the DAPHNE consortium as well as external contributors (mentioned below). Find some of the more notable (bigger) changes below in the detailed release notes. Besides the artifact downloads on this page and the Docker images, we are releasing the python bindings daphne-lib on PyPI
Release Highlights
- DaphneDSL/DaphneLib:
- Additional ready-to-use data science algorithms: decision trees, random forests, PageRank
- Various DaphneDSL language improvements (e.g., UDFs with multiple return values, more built-in functions, bounds-checks for left/right indexing, ...)
- Support for complex control flow in DaphneLib
- DaphneLib as a Python package
- Efficient data exchange with pandas, TensorFlow, PyTorch
- DAPHNE Compiler:
- Initial MLIR-based codegen pipeline
- Introduction of a kernel catalog to make the compiler aware of available pre-compiled kernels
- DAPHNE Runtime:
- CUDA 12.2.2
- CUDA-support for more kernels
- Support for synchronous gRPC and chunked data transfer in distributed runtime
AUTO
option to the DAPHNE’s scheduling algorithms
- Infrastructure and general:
- Initial extensibility for custom kernels
- More consistent and actionable error messages
- Numerous little improvements and bug fixes
Contributors to this release
- AlexRTer
- Aristotelis Vontzalidis
- Benjamin Steinwender
- Constantin Pestka
- DamianDinoiu
- Daniel Wetzel
- Garic
- Henri Willems
- Jonas Henrique Muller Korndorfer
- Lachezar Nikolov
- Lorenz Dirry
- Marcus Paradies
- Marius Birkenbach
- Mark Dokter
- Patrick Damme
- Philipp Ortner
- Quentin Guilloteau
- Samin
- StoeckOverflow
- Stratos Psomadakis
- Tom Schwarzburg
- inikokali
Instructions to download and verify
# The signing keys
wget -qO- https://raw.githubusercontent.com/daphne-eu/daphne/main/KEYS.txt | gpg --import
# The plain DAPHNE release (ARMv8 aka 64 bit ARM)
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-ARMV8-0.3-bin.tgz.sha512sum
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-ARMV8-0.3-bin.tgz.asc
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-ARMV8-0.3-bin.tgz
sha512sum -c daphne-ARMV8-0.3-bin.tgz.sha512sum
gpg --verify daphne-ARMV8-0.3-bin.tgz.asc
tar xf daphne-ARMV8-0.3-bin.tgz
# add lib directory to environment
export LD_LIBRARY_PATH=$PWD/daphne-ARMV8-0.3-bin/lib:$LD_LIBRARY_PATH
# The plain DAPHNE release (X86-64)
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-X86-64-0.3-bin.tgz.sha512sum
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-X86-64-0.3-bin.tgz.asc
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-X86-64-0.3-bin.tgz
sha512sum -c daphne-X86-64-0.3-bin.tgz.sha512sum
gpg --verify daphne-X86-64-0.3-bin.tgz.asc
tar xf daphne-X86-64-0.3-bin.tgz
# add lib directory to environment
export LD_LIBRARY_PATH=$PWD/daphne-X86-64-0.3-bin/lib:$LD_LIBRARY_PATH
# The DAPHNE release with CUDA support (X86-64)
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-cuda-X86-64-0.3-bin.tgz.sha512sum
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-cuda-X86-64-0.3-bin.tgz.asc
wget https://github.com/daphne-eu/daphne/releases/download/0.3/daphne-cuda-X86-64-0.3-bin.tgz
sha512sum -c daphne-cuda-X86-64-0.3-bin.tgz.sha512sum
gpg --verify daphne-cuda-X86-64-0.3-bin.tgz.asc
tar xf daphne-cuda-X86-64-0.3-bin.tgz
# add lib directory to environment
export LD_LIBRARY_PATH=$PWD/daphne-cuda-X86-64-0.3-bin/lib:$LD_LIBRARY_PATH
What's Changed
- Fix erroneous special treatment of default logger by @MarcusParadies in #594
- Cc cuda ops by @corepointer in #596
- Misc improvements by @corepointer in #604
- Fix passing str variable to readFrame/readMatrix by @MarcusParadies in #598
- [DAPHNE-#554] Use hwloc to query CPU topology by @psomas in #563
- [doc] add more docs regarding script arguments by @m-birke in #620
- [DAPHNE-#606] Some INT/FLOAT literal suffixes from C++ in DaphneDSL by @corepointer in #606
- 522 DaphneLib complex control flow and lazy evaluated functions by @lachezar-n in #574
- 615-Constant_folding_EwlogOp by @inikokali in #630
- [MINOR] Enable shape inference for frame type by @DamianDinoiu in #626
- [MINOR] Parsing IR fix. by @aristotelis96 in #634
- MLIR-based code generation pipeline by @philipportner in #633
- SQLParser: Introduce DISTINCT clause by @tomschw in #564
- SQLParser: Support for SELECT , SELECT f., GROUP BY , GROUP BY f. by @tomschw in #588
- 613-Support missing aggregation functions in aggregation kernels by @inikokali in #638
- Integration of mpi_testing into test.sh by @inikokali in #639
- Close #614 Support missing elementwise unary functions in elementwise unary kernels by @AlexRTer in #644
- [doc] fix typo in links within Readme by @AlexRTer in #645
- Automated task grain size calculation (related to issue 567). by @jhmkorndorfer in #647
- #650 fix get topology by @GuilloteauQ in #651
- [DAPHNE-#572] Bounds checks for left and right indexing by @AlexRTer in #648
- Distributed worker: Insert IR arguments to body. by @aristotelis96 in #623
- [DAPHNE-#669] properly handle views in
WriteCsv
kernel by @AlexRTer in #678 - Feature/daphnelib python pkg by @m-birke in #649
- Pandas, Pytorch & Tensorflow Shared Memory Support (Issue #499) by @danielwetzel in #585
- [DAPHNE-#687] OneHot bound check by @AlexRTer in #695
- Code Generation for Matrix Multiplication by @resting-dove in #653
- Improve error handling during compilation + runtime by @philipportner in #706
- Basic extension catalog for kernels, plus demo on early extensibility. by @pdamme in #456
- Generic Matrix kernel specializations by @AlexRTer in #722
- [BUGFIX] Build target dependencies. by @pdamme in #728
- [DAPHNE-#531] Simplified GettingStarted.md. by @pdamme in #725
- [DAPHNE-#659] Scientific notation for float literals and digit separators by @AlexRTer in #677
- Tensor Support - 1: Data structure by @CPestka in #694
- Non-deterministic test fail during Github CI action by @philipportner in #733
- Enable MathJax for documentation by @AlexRTer in #726
- [DAPHNE-#747] Use test.sh in CI again. by @pdamme in #748
- [DAPHNE-#523] DaphneDSL flexible matrix/frame literals by @AlexRTer in #675
- daphnelib: improve pyproject.toml and readme by @m-birke in #743
- Update docs.yml by @auge in #750
- Update main.yml by @auge in #751
- Add missing (local) kernel tests by @AlexRTer in #730
- [DAPHNE-#658] stop() built-in/op/kernel by @Garic152 in #753
- [DAPHNE-#666] optional inc for seq function by @saminbassiri in #756
- mac support ongoing by @auge in #764
- 752 one hot and bin functions in daphne lib by @saminbassiri in #760
- [DAPHNE-#680] bug fix for signed integer casts by @AlexRTer in #744
- [security] Remove polyfill.io dependency in docs by @AlexRTer in #769
- #720 replace asserts in codebase with exceptions by @AlexRTer in #732
- Add AggAll kernels for missing value types by @resting-dove in #674
- Address Wrong results using Integer Matmul Kernels after Matrix resizing by @resting-dove in #701
- Refactor getLine() to avoid potential memory leak by @psomas in #770
- [DAPHNE-#560] Parsing unary minus operator by @corepointer in #607
- [DAPHNE-#595] Fix crash when calling print(f()) with empty function by @corepointer in #605
- [DAPHNE-#768] isNan: elementwise check for NaN elements in a matrix - Draft PR by @StoeckOverflow in #779
- Update software requirements in doc by @AlexRTer in #786
- 657 DaphneLib implement indexing by @saminbassiri in #771
- Implementation of new SparsityOp by @Garic152 in #790
- Improve performance of the transposition of CSR matrices by @GuilloteauQ in #798
- 788 closing mem leaks by @corepointer in #791
- [DAPHNE-#774] Simplification rewrites for elementwise unary minus (ad… by @ldirry in #796
- [DAPHNE-#775] Support for unary minus in DaphneLib by @ldirry in #792
- [DAPHNE-#755] Initial support for lists in DaphneDSL. by @pdamme in #806
- Implementation of CSR/CSR Matrix Multiplication (
@
) by @GuilloteauQ in #799 - [DAPHNE-#773] Undetected invalid script args by @ldirry in #800
New Contributors
- @MarcusParadies made their first contribution in #594
- @lachezar-n made their first contribution in #574
- @inikokali made their first contribution in #630
- @AlexRTer made their first contribution in #644
- @jhmkorndorfer made their first contribution in #647
- @GuilloteauQ made their first contribution in #651
- @danielwetzel made their first contribution in #585
- @resting-dove made their first contribution in #653
- @CPestka made their first contribution in #694
- @Garic152 made their first contribution in #753
- @saminbassiri made their first contribution in #756
- @StoeckOverflow made their first contribution in #779
- @ldirry made their first contribution in #796
Full Changelog: 0.2...0.3