Skip to content

Commit 0c520f8

Browse files
committed
Clean up GPU support
Signed-off-by: Whelan Boyd <whelanboyd@gmail.com>
1 parent 6a01cf7 commit 0c520f8

File tree

1 file changed

+8
-1
lines changed

1 file changed

+8
-1
lines changed

src/content/blog/december-2025.mdx

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,14 @@ published: true
1111
The work mentioned in the previous bulletin to revamp Array evaluation to be fully lazy was released on January 6th. This happens by converting their execution to an Operator model that evaluates into Vectors (fully decompressed, zero-copy to Arrow representation). As a reminder, this work enables many more optimizations, and also provides unified abstractions for evaluating on
1212
different processor types (CPUs & GPUs).
1313

14-
Speaking of, focus is now on GPU support. The goal is to enable querying training data on the fly and streaming it from object storage directly to GPU memory with high throughput. To achieve this, the team is: - adding the necessary encodings via Vortex's extensions to enable GPU-native decompression - integrating with NVIDIA's CUDA toolkit so GPUs can scan Vortex files in object storage directly
14+
## GPU Support
15+
16+
Speaking of, focus is now on enabling reading Vortex files into GPUs. To achieve this, the team is:
17+
18+
- Adding GPU decompression for existing kernels, as well as some new encodings optimized for GPU
19+
- Integrating with NVIDIA's CUDA toolkit for high performance I/O.
20+
21+
The first supported output types will be Arrow Device Arrays and cuDF. As with all of Vortex, these capabilities are fully exposed to plugins so advanced users can extend and customize for their own use.
1522

1623
## Around the Ecosystem
1724

0 commit comments

Comments
 (0)