blog post v1.0.7

website/blog/releases/v1.0.7.md

@@ -0,0 +1,123 @@
+---
+date: 2025-03-27
+title: 'Spice v1.0.7 (Mar 26, 2025)'
+type: blog
+authors: [phillipleblanc]
+tags: [release, duckdb, arrow, dremio]
+---
+
+Announcing the release of Spice v1.0.7 🏎️
+
+Spice v1.0.7 improves memory usage when using DuckDB, improves schema inference performance when using object-store based data connectors, and fixes a bug in Dremio schema inference.
+
+## Highlights in v1.0.7
+
+- **DuckDB Memory Usage**: Memory usage when using DuckDB has been significantly improved for data loads and refreshes through expanded use of zero-copy Arrow and multi-threading for data loads. When a `duckdb_memory_limit` is specified, disk spilling has been improved for greater-than-memory workloads. In addition, a new `temp_directory` runtime parameter supports storing temporary files to alternative location than the DuckDB data file for higher throughput. For example, `temp_directory` could be set to a different high-IOPs IO2 EBS volume that is separate from the `duckdb_file_path`.
+
+  Automated end-to-end tests for the DuckDB Accelerator coverage has been [significantly expanded](https://github.com/spiceai/spiceai/pull/5146).
+
+  For configuration details, see the documentation for [runtime parameters](https://spiceai.org/docs/reference/spicepod#runtime) and the [DuckDB Data Accelerator](https://spiceai.org/docs/components/data-accelerators/duckdb).
+
+- **Schema Inference Performance for Object-Store Data Connectors**: Schema inference performance has been improved, especially for large numbers of objects (1M+ objects) when using object-store based data connectors by making the object-listing and selection more efficient.
+
+## Performance
+
+When compared to previous versions, Spice v1.0.7 loads DuckDB accelerated datasets significantly faster. When using the TPCH lineitem dataset at Scale Factor 100 (600M rows):
+
+### Without Indexes
+
+5x faster, 28% less memory usage.
+
+![v1.0.6](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/68efa3c6-caa8-447d-25ee-5081e4c47800/public)
+![v1.0.7](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/f25b66cd-1826-4f3d-6233-466228858300/public)
+
+| Version | Load Time | Peak Memory Usage |
+| ------- | --------- | ----------------- |
+| v1.0.6  | 16m 3s    | 32GB              |
+| v1.0.7  | 3m 149ms  | 24.4GB            |
+
+### With Indexes
+
+2.5x faster. Higher memory usage in v1.0.7 is due to better resource utilization to achieve faster load times. Use the `duckdb_memory_limit` parameter to control memory usage.
+
+| Version | Load Time | Peak Memory Usage |
+| ------- | --------- | ----------------- |
+| v1.0.6  | 27m 9s    | 50GB              |
+| v1.0.7  | 11m 30s   | 77GB              |
+
+![v1.0.6 with indexes](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/104e7dcf-5f85-47ee-ad6c-7417b0aa8800/public)
+![v1.0.7 with indexes](https://imagedelivery.net/HyTs22ttunfIlvyd6vumhQ/402c41b4-d81f-4ef4-8878-a06da6916f00/public)
+
+## Documentation
+
+- [DuckDB Data Accelerator](https://spiceai.org/docs/components/data-accelerators/duckdb): Has been expanded with additional resource usage guidance.
+- [Memory](https://spiceai.org/docs/reference/memory): A new section for memory considerations has been added to the Reference section.
+
+## Contributors
+
+- @phillipleblanc
+- @sgrebnov
+- @peasee
+- @Sevenannn
+
+## Breaking Changes
+
+No breaking changes.
+
+## Upgrading
+
+To upgrade to v1.0.7, use one of the following methods:
+
+**CLI**:
+
+```console
+spice upgrade
+```
+
+**Homebrew**:
+
+```console
+brew upgrade spiceai/spiceai/spice
+```
+
+**Docker**:
+
+Pull the spiceai/spiceai:1.0.7 image:
+
+```console
+docker pull spiceai/spiceai:1.0.7
+```
+
+For available tags, see [DockerHub](https://hub.docker.com/r/spiceai/spiceai/tags).
+
+**Helm**:
+
+```console
+helm repo update
+helm upgrade spiceai spiceai/spiceai
+```
+
+## What's Changed
+
+### Dependencies
+
+- [DataFusion Table Providers](https://github.com/datafusion-contrib/datafusion-table-providers): Upgraded from `760ece6ac52b7d180d697f347642af403c2e711c` to `9ba9dce19a1fdbd5e22cc2e445c5b3ea731944b4`.
+
+### Changelog
+
+```text
+- fix: Remove on zero results arguments from benchmarks by @peasee in https://github.com/spiceai/spiceai/pull/4533
+- Run benchmark tests w/o uploading test results (pending improvements) by @sgrebnov in https://github.com/spiceai/spiceai/pull/4843
+- fix: Return BAD_REQUEST when not embeddings are configured by @peasee in https://github.com/spiceai/spiceai/pull/4804
+- Fix Dremio schema inference by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5114
+- Improve performance of schema inference for object-store data connectors by @sgrebnov in https://github.com/spiceai/spiceai/pull/5124
+- Always download spice runtime version matched with spice cli version by @Sevenannn in https://github.com/spiceai/spiceai/pull/4761
+- Fix go lint errors by @sgrebnov in https://github.com/spiceai/spiceai/pull/5147
+- Make DuckDB acceleration E2E tests more comprehensive by @sgrebnov in https://github.com/spiceai/spiceai/pull/5146
+- Enable Spice to load larger than memory datasets into DuckDB accelerations by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5149
+- Add `temp_directory` runtime parameter and insert it for DuckDB accelerations by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5152
+- Fix Postgres and MySQL installation on macos14-runner (E2E CI) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5155
+- Enable E2E for DuckDB full mode acceleration with indexes only in CI by @sgrebnov in https://github.com/spiceai/spiceai/pull/5154
+```
+
+**Full Changelog**: [github.com/spiceai/spiceai/compare/v1.0.6...v1.0.7](https://github.com/spiceai/spiceai/compare/v1.0.6...v1.0.7)

website/docusaurus.config.ts

@@ -54,6 +54,7 @@ const config: Config = {
           path: 'blog',
           showLastUpdateAuthor: true,
           showLastUpdateTime: true,
+          onUntruncatedBlogPosts: 'ignore',
           editUrl: ({ locale, blogDirPath, blogPath }) => {
             return `https://github.com/spiceai/docs/edit/trunk/website/${blogDirPath}/${blogPath}`
           },
@@ -91,8 +92,7 @@ const config: Config = {
     // Replace with your project's social card
     image: 'img/og/spiceai.png',
     announcementBar: {
-      content:
-        '<a href="/blog/releases/v1.0.7">Spice.ai OSS v1.0.7</a> is now available! 🚀',
+      content: '<a href="/blog/releases/v1.0.7">Spice.ai OSS v1.0.7</a> is now available! 🚀',
       backgroundColor: 'var(--announcement-bar-bg)',
       textColor: 'var(--announcement-bar-text)',
       isCloseable: true