Tags: modin-project/unidist
Tags
unidist 0.7.1 This release contains an important bugfix for MPI backend using shared memory. Key Features and Updates ------------------------ * Stability and Bugfixes * FIX-#425: Fix type of shared MPI.buffer (#426) Contributors ------------ @Retribution98
unidist 0.7.0 This release makes wheels for PyPI valid, as well as upgrade minimal supported python version to 3.9. Key Features and Updates ------------------------ * Stability and Bugfixes * FIX-#416: Modify setup.py to enable python and platforms specific builds (#417) * FIX-#416: Add missing files into sdist (#418) * FIX-#416: Up minimal python version to 3.9 (#420) * FIX-#416: Build cython extension with c++11 support (#421) * FIX-#416: Upgrade versioneer to get correct version on Windows (#422) * FIX-#416: Build bdist and sdist in CI and upload to PyPI (#423) Contributors ------------ @YarShev
unidist 0.6.0 This release contains a few important improvements for MPI backend. Key Features and Updates ------------------------ * Stability and Bugfixes * FIX-#402: Fix MPICH version check (#403) * FIX-#404: Add psutil as a required dependency for unidist[mpi] (#406) * FIX-#407: Make data put into MPI object store immutable (#409) * Performance enhancements * PERF-#360: Re - Implement asyncronous get request with a fix (#405) * Refactor Codebase * REFACTOR-#000: Refactor remote function class for MPI backend (#410) Contributors ------------ @YarShev @Retribution98 @arunjose696
unidist 0.5.1 This release contains an important bugfix for MPI backend and adds psutil as a required dependency for unidist[mpi] package. Key Features and Updates ------------------------ * Stability and Bugfixes * FIX-#402: Fix MPICH version check (#403) * FIX-#404: Add psutil as a required dependency for unidist[mpi] (#406) Contributors ------------ @YarShev @Retribution98
unidist 0.5.0 This release adds support for a shared object store for MPI backend, which can be enabled with UNIDIST_MPI_SHARED_OBJECT_STORE environment variable, contains a lot of performance enhancements, includes several features, important documentation updates and a bunch of bugfixes. Key Features and Updates ------------------------ * Stability and Bugfixes * FIX-#329: Allow remaining communication requests to complete (#330) * FIX-#343: Get rid of the hacks in the MPI backend (#344) * FIX-#345: Test both modes of MPI execution on Linux only (#346) * FIX-#349: Remove the warning regarding MpiPickleThreshold (#350) * FIX-#354: Fix unidist.wait for MPI backend (#355) * FIX-#363: Fix DataID usage between different MPI processes (#364) * FIX-#371: Fix check for out of bounds reserved memory in shared storage (#372) * FIX-#375: Fix running an MPI cluster without using shared memory (#377) * FIX-#340: Fix hangs at low UNIDIST_MPI_SHARED_OBJECT_STORE_THRESHOLD (#380) * FIX-#376: Partition buffers of size larger than the _bigmpi.blocksize into blocks while sending and receiving with MPI (#383) * FIX-#337: Fix MpiHosts parameter usage (#378) * FIX-#000: Revert PR #387 (#388) * FEAT-#392: Raise a warning if shared object storage is enabled but the MPI library does not support it (#393) * FIX-#389: Revert async get changes made in #359 (#390) * FIX-#394: Fix CI running on Windows (#395) * FIX-#396: Fix Split_type call for Unidist on MSMPI (#397) * Performance enhancements * PERF-#365: Cache info about sent metadata of shared data (#366) * PERF-#360: Implement asyncronous get request (#359) * PERF-#367: Use std::fill to fill service shared buffer with a given value (#373) * New Features * FEAT-#285: Implement shared object storage for MPI backend (#286) * FEAT-#341: Add the ability to specify environment variables for MPI workers (#352) * FEAT-#347: Get rid of mpi4py-mpich dependency (#358) * FEAT-#386: Make MPI shared object store default option (#387) * Refactor Codebase * REFACTOR-#000: Remove unnecessary build step in CI (#339) * REFACTOR-#000: Introduce task_info metadata package (#348) * REFACTOR-#000: Remove unncessary else branch in mpi init (#351) * REFACTOR-#000: Change mpi_state.comm to mpi_state.global_comm (#379) * REFACTOR-#000: Change IsMpiSpawnWorkers to MpiSpawn to make it more concise (#382) * Documentation improvements * DOCS-#000: Update the workflow status badge (#332) * DOCS-#000: Add the downloads badge in readme (#333) * DOCS-#384: Fix warnings when building documentation (#385) * DOCS-#336: Add documentation about shared object storage (#391) Contributors ------------ @YarShev @Retribution98 @arunjose696
unidist 0.4.0 This release makes MPI the default backend, includes new features and important documentation updates for it, as well as contains a bunch of bugfixes. Key Features and Updates ------------------------ * Stability and Bugfixes * FIX-#275: Pass python flags into workers (#276) * FIX-#281: Do not shutdown MPI backend multiple times (#282) * FIX-#283: Exit monitor and worker processes in SPMD mode (#284) * FIX-#294: Fix MPI backend launch for Open MPI in a cluster (#295) * FIX-#242: Fix behavior of unidist.wait for MPI backend (#291) * FIX-#000: Add licence header to request_store.py (#307) * FIX-#296: Make workers exit the loop correctly (#312) * FIX-#318: Do not initialize MPI in serialization.py (#319) * FIX-#320: Improve CI (#321) * FIX-#322: Disable actor tests for Ray (#323) * FIX-#324: Pin pydantic<2 to fix CI (#325) * FIX-#326: Set I_MPI_SPAWN for Intel MPI when using dynamic spawn (#327) * Performance enhancements * PERF-#287: Optimize python list appends (#290) * New Features * FEAT-#288: Make MPI as default backend (#293) * FEAT-#305: Introduce MpiBackoff configuration value (#306) * FEAT-#316: Introduce MpiLog configuration value (#317) * Refactor Codebase * REFACTOR-#000: Refactor request store to make it clearer (#311) * Documentation improvements * DOCS-#000: Clarify the phrases regarding pyseq backend (#272) * DOCS-#277: Add notes regarding installation of MPI implementations (#278) * DOCS-#279: Advise original MPI binaries rather than conda-forge packages (#280) * DOCS-#299: Extend troubleshooting page as to MPI errors (#300) * DOCS-#301: Add Optimization Notes section to the docs (#302) * DOCS-#303: Fix a typo in unidist_on_mpi.rst (#304) * DOCS-#314: Add troubleshooting block for 'OpenSSL version mismatch' error (#315) Contributors ------------ @YarShev @Retribution98 @arunjose696 @luweizheng
unidist 0.3.0
This release contains a few performance enhancements for MPI backend,
the renaming of backends "multiprocessing" to "pymp" and "python" to "pyseq",
includes a couple of important updates for documentation and multiple bugfixes.
Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-#170: Fix serialization/deserialization for Ray actors (#171)
* FIX-#172: Fix serialization/deserialization for Dask actors (#173)
* FIX-#174: Fix serialization/deserialization for MPI actors (#177)
* FIX-#179: Handle None return values correctly on MPI backend (#180)
* FIX-#175; FIX-#178: Fix nested remote calls of remote tasks and actor methods (#176)
* FIX-#187: Make Data_ID unique for different workers on MPI backend (#188)
* FIX-#183: Fix serialization of dict_keys for MPI backend (#184)
* FIX-#189: Fix Actor scheduling (#190)
* FIX-#191: Fix pendinq response after blocking GET request (#192)
* FIX-#193: Fix actor async execution (#194)
* FIX-#197: Initialize a backend when doing unidist.put (#198)
* FIX-#201: Fix warnings to be aligned with latest flake8 (#202)
* FIX-#211: Fix dask[complete] syntax in conda environment file (#212)
* FIX-#213: Fix NamedTuple serialization (#215)
* FIX-#209: Fix set up of conda in CI to make it work properly (#210)
* FIX-#216: Fix development environment for ray==2.1.0 (#217)
* FIX-#218: Install Ray from PyPI to make CI work (#220)
* FIX-#224: Fix recursion in processing remote task arguments and serialization of dictionary that has a tuple key (#225)
* FIX-#228: Add missing configs to workers initialization (#229)
* FIX-#233: Fix passing MpiHosts config to workers initialization (#234)
* FIX-#232: Fix sending out-of-band data over 1 GB (#231)
* FIX-#235: Adding handler only if logger does not have any handlers (#236)
* FIX-#246: Slight changes to file formatting to match new version of black (#247)
* FIX-#253: Append worker rank to logger name (#254)
* FIX-#256: Fix sending a cleanup command from a worker to itself (#257)
* FIX-#263: Update the year in the licence header (#264)
* FIX-#259: Fix cleanup data in the workers (#260)
* FIX-#269: Update min supported Ray version (#270)
* Performance enhancements
* PERF-#226: Do not use pickle5 for small object serialization (#227)
* PERF-#243: Using non-blocking send (#244)
* PERF-#195: Fix serialization for numpy objects (#200)
* PERF-#195: Fix serialization for pandas and numpy objects (#196)
* New Features
* FEAT-#185: Add communication logging (#186)
* FEAT-#204: Allow PERF prefix in the commit messages (#205)
* FIX-#248: Rename backends "multiprocessing" to "pymp" and "python" to "pyseq" (#265)
* Refactor Codebase
* REFACTOR-237: Replace len_buffers with buffer_count (#238)
* Documentation improvements
* DOCS-#207: Add a doc section regarding installation of Intel MPI (#208)
* DOCS-#266: Fix documentation build (#267)
* DOCS-#221: Add mentioning about Modin and unidist integration in to docs (#222)
* Update testing suite
* TEST-#181: Added async actor tests (#182)
* TEST-#89: Set up testing for unidist on push (#206)
* FIX-#250: Require format for PR descriptions instead of commit descriptions (#251)
* FIX-#250: Replace lint-commit to check-pr-title in ci.yml (#252)
Contributors
------------
@YarShev
@Retribution98
@arunjose696
unidist 0.2.2
This release contains a few important bugfixes for MPI backend.
Key Features and Updates
------------------------
* Stability and Bugfixes
* FIX-#201: Fix warnings to be aligned with latest flake8 (#202)
* FIX-#195: Fix serialization for numpy objects (#200)
* FIX-#211: Fix dask[complete] syntax in conda environment file (#212)
* FIX-#213: Fix NamedTuple serialization (#215)
* FIX-#224: Fix recursion in processing remote task arguments and serialization of dictionary that has a tuple key (#225)
* FIX-#228: Add missing configs to workers initialization (#229)
Contributors
------------
@YarShev
@Retribution98
PreviousNext