Release notes from deeplake

v4.4.4

2025-12-24T22:18:28Z

Deeplake API now has cmake and pkg-config files for better integration
Improved storage access performance
Revisited PG ingestion. Improved batch ingestion

v4.4.3

2025-12-04T17:06:45Z

First C++ library release.

v4.4.1 🌈

2025-11-21T10:52:52Z

Release Notes

🎉 New Features & Enhancements

Storage & API Improvements

Storage list_dirs API - New API endpoint for listing directories by @khustup2
Added mesh type support - Extended data type capabilities by @khustup2
PLY visualization and local file listing - Enhanced node server with visualization tools by @azat-manukyan
Simple visualizer - New visualization component for data inspection by @khustup2
Credentials key property - Added property to expose creds key for read-only datasets by @activesoull

Data Management

Efficient version-locked read-only view - Implemented optimized read-only dataset views by @ashotshakhkyan
Library preloading - Improved performance with preloaded library support by @vahan-activeloop

⚡ Performance Improvements

30% LRU cache insertion runtime improvement - Significant speedup in cache operations by @ashotshakhkyan
Cache clearing on dataset deletion - Minor performance optimization on cleanup by @ashotshakhkyan
Refactored async_prefetcher - Simplified batch prefetch logic for better performance by @ashotshakhkyan

🐛 Bug Fixes

Core Functionality

Fixed progress reporting of rebuild_branch - Corrected progress tracking during branch rebuilds by @khustup2
Fixed deeplake behavior - Resolved critical deeplake module issues by @activesoull
Fixed JS build - Resolved JavaScript compilation errors by @ashotshakhkyan

Data Handling

Summary fix - Corrected summary generation logic by @activesoull
MP3 header magic bytes - Fixed audio file format detection by @azat-manukyan

Indexing & Storage

Corrected indexing mode of compactor - Fixed compactor indexing behavior by @ashotshakhkyan
Duplicate index data handling - Resolved race condition in concurrent index builders by @ashotshakhkyan
Fixed warning - Addressed compiler warnings by @activesoull

🏗️ Architecture & Refactoring

Code Organization

Simplified rebuild_branch logic - Eliminated temporary branch creation for cleaner code by @khustup2
Reduced public headers - Hidden implementation details from public API by @ashotshakhkyan
Deeplog include cleanup - Organized include statements for better maintainability by @ashotshakhkyan

API Improvements

Added merge commit message - Enhanced commit metadata by @activesoull

v4.3.4 🌈

2025-10-10T17:49:30Z

🎉 New Features
pg_deeplake Enhancements

CHAR/BPCHAR data type support - pg_deeplake now supports CHAR and BPCHAR data types
Library preloading - Improved performance with preloaded libraries in pg_deeplake
Auto table reload - Tables are now automatically reloaded if not found in storage

Architecture Improvements

Query core module - Separated query engine from execution module for better modularity
Group by execution refactor - Separated group by execution logic for future enhancements

🐛 Bug Fixes

Fixed linked audio type data handling - Resolved issues with linked audio type data
Fixed compilation errors - Resolved compile errors in pg_deeplake

💬 User Experience

Better error messaging - Added clear error messages for missing credentials key

🧹 Code Quality

Cleanup and refactoring - Improved code organization with proper renames and cleanup

🧪 Testing

Enhanced test coverage - Added comprehensive tests for pg_deeplake

v3.9.52 🌈

2025-09-05T13:50:53Z

🧭 What's Changed

Allow numpy v2. (#3075) @khustup2

⚙️ Who Contributes

@activesoull and @khustup2

v4.4.0

2025-10-24T19:03:18Z

Release Notes

New Features

Indexing Improvements

Support for JSON data indexing - Added support for indexing JSON data types with proper handling and query optimization
Generic column index specification - New ability to specify index type on generic columns for more granular control
Progress bar for index creation - Improved user experience with visual progress indicator during index creation operations
Index handling unification - Streamlined and unified index handling across the codebase for better consistency and maintainability

Documentation

Added missing documentation sections - Expanded documentation coverage for previously undocumented features and APIs

Improvements

Performance & Reliability

Fixed build errors on main branch - Resolved compilation issues affecting the main development branch
Windows test fixes - Enhanced test reliability on Windows platform with targeted fixes
Thread-safe cancellation - Improved cancellation mechanism with proper thread safety guarantees
Rebuild branch internal API - Refactored internal API for better architecture and maintainability
Fix rebuild indexes during compaction - Corrected index rebuilding behavior during database compaction operations

Platform & Build Support

Windows Python build support - Added comprehensive support for building Python bindings on Windows
Updated libtiff dependency - Upgraded libtiff library to latest version for improved compatibility and security

Bug Fixes

Fixed browser crash - Resolved critical issue causing browser crashes in specific scenarios
Invalid iterator fix - Fixed invalid iterator handling that could cause runtime errors
General bug fixes - Various stability improvements and edge case handling

Contributors

Thanks to @activesoull, @vahan-activeloop, @khustup2, and @ashotshakhkyan for their contributions to this release.

v4.3.5 🌈

2025-10-10T17:50:47Z

Data Type Support

Link to bytes support - Added support for link to bytes data type (DEE-299)
Flexible text column handling - Text columns now allow appending data with unknown dtype for better compatibility

🐛 Bug Fixes

deeplake.exists error handling - Fixed exception handling in deeplake.exists to gracefully handle errors instead of throwing exceptions (DEE-300)
Python PNG linking - Resolved Python PNG linking issues

📚 Documentation

Documentation updates - Improved and updated documentation (DEE-36)

🧹 Code Quality

General improvements - Various small fixes and code improvements

v4.3.4

2025-10-10T15:16:47Z

🎉 New Features
PostgreSQL 18 Support

PostgreSQL 18 compatibility - pg_deeplake now supports PostgreSQL 18

Performance Optimizations

Adaptive backoff for worker threads - Implemented an intelligent backoff strategy for idle worker threads to reduce CPU usage
Cgroup-aware concurrency - Hardware concurrency now respects cgroup limits for better container performance

Azure Integration

Azure container name handling - Improved Azure getter with proper container name handling

🐛 Bug Fixes

Build errors resolved - Fixed compilation errors in pg_deeplake
Visualizer fixes - Resolved issues in the visualizer component

🧹 Code Quality

Code cleanup - Improved code organization with proper renames and refactoring
Modular architecture - Continued separation of concerns for better maintainability

🧪 Testing

Enhanced test coverage - Added comprehensive tests for pg_deeplake

v4.3.0 🌈

2025-08-29T08:50:37Z

Deeplake 4.3.0

Deeplake 4.3.0 is a major update bringing many new features to the Deeplake ecosystem.

New Data and Index Types

Complete revisit of Sequence types to support visual and structured data
Video type support is now available in Deeplake, supporting MP4 and MKV videos with H264 codec and providing fast random access to video frames
Indexing for numeric types, enabling fast queries for numeric comparisons in TQL, including IN and BETWEEN operations
Significant improvements to textual index types, providing faster search without requiring index regeneration

Data Import/Export

Fully rewritten from_csv function with support for large CSV files
New to_csv API to export Deeplake datasets/views to CSV format

Python Typing

Support for specifying Python builtin types when defining dataset schemas
Support for using Pydantic Models as dataset schemas
Enriched async operations typing, to support better integration with linters and IDEs.

Improvements and Bug Fixes

Improved TQL data fetching and linear scan performance for non-indexed columns
Better memory usage tracking to prevent out-of-memory errors
Various stability improvements and bug fixes

Compatibility Notice

Deeplake 4.3.0 is backward compatible with datasets created in v4.2.x. However, datasets created or modified with v4.3.0 cannot be opened with v4.2.x versions due to internal format enhancements. We recommend upgrading all environments to v4.3.0 when working with shared datasets.

v3.9.51 🌈

2025-08-07T07:23:51Z

🧭 What's Changed

Update version for 3.9.51 release (#3073) @activesoull
[DEE-180] fix dependency resolver issue (#3072) @activesoull

⚙️ Who Contributes

@activesoull